Scandal! + UI (AI?) Accessibility

Good morning.

Good morning, Drew. Good morning, everyone. Welcome to Drew and Dan in the morning. This is the podcast with Drew and Dan. We're ⁓ where we talk about what we are random things. We're still trying to figure out what we talk about, but ⁓ it's definitely

Good morning, Dan. Random things. Well, I know what I need to talk about. We need to talk about this scandal with China. This is probably I I don't know if every app developer is aware of this, ⁓ in the US or in Europe, but ⁓ it is a huge, huge, huge scandal. And honestly, I didn't see anything about this in the news. I didn't see about it on like The Verge or ⁓ what's that other little podcast a couple people listen to who's

Yeah. Techbrew ride home?

TechPro right home. I hear it on there. didn't hear it on the ⁓ accidental tech podcast. I didn't hear it anywhere. The scandal is this. Apparently, back in March, very secretly, Apple reduced all their commission rates for I'm sorry, what is the actual word? Is it if your base currency is China, your base store is China or something? Something like that. But basically, if you're in China, you get to pay less commission than you do here in the US, which is really annoying because ⁓

What is the scandal?

Come on, guys. ⁓ I know I pay Apple every month, what, $30,000, $40,000 as their ⁓ 15%. ⁓ no, no, no, I'm sorry. $30 or $40 every some period of time. Let's call it that. ⁓ yeah. So the Apple Developer Program License Agreement has reduced the fee from 30% to 25% if you're in China. Or if you're at 15%. So on

This is a ficti fictitious sh show. Decade.

You know, your small business program or your Second year of a subscription. when you're normally at fifteen percent, you are now at twelve percent. So they lowered it five percent on the full price and three percent on the lower price. And I think it's a scandal. I cannot believe no one's talked about this. As much as people get in arms up at arms about, ⁓ my gosh, Elb's charging us a lot of money.

Right, right. Yeah. I I I I didn't see anything about this either. Or or maybe I caught a hint of it, but it went right over the top of my head because it doesn't really apply to me because I make so little money anyway. like a as in I make almost no money, ⁓ a a dollar or three every month or so. But ⁓

Dan, what what have you heard about this? Have you heard anything?

⁓ I think that for a a good size developer, ⁓ indie developer, or or these large apps, ⁓ a a drop of three percent or five percent on the commission is a lot. ⁓ maybe we haven't heard about it because it is isolated to China and and because of some court ruling. But ⁓ whether or not that filters over to the rest of the app store or the rest of the world, well, I guess that remains to be seen. But ⁓ it is it's an interesting thing to keep an eye on. Anybody would love ⁓ for an additional five percent off.

Yeah. Well, I just ⁓ honestly, I mean, obviously it's China, I don't really care. So, 'cause I'm not in China. But also I really honestly can't believe these other ⁓ people haven't picked up on it and made a big stink because they've spent so ⁓ so many people have spent so much energy making a stink about Apple's cut on things that I mostly don't care about. but i th it just shocks me that that's not ⁓ not come up as a thing. But ⁓ anyway, just discovered that today. So that's my

Right. Something to keep an eye on.

My news item of the day. ⁓ got a couple other others. ⁓ let's see, the second one here. ⁓ yeah, ⁓ I did look at the app review guidelines recently and I had I was working on a little project that goes through and like points out when things have changed and so you can compare like, ⁓ what's the version today versus the version on October sixteenth of twenty twenty three and then see what's changed between them or whatever. Anyway, the one thing I happen to notice, and ⁓ this has changed sometime since February. ⁓ because I hadn't run my little update script in a while. ⁓ under section 1.2 user generated content, ⁓ apps with user generated content are services that end up being used primarily for pornographic content, chat style root ⁓ chat root can't talk, chat roulette style experiences, and then this is the new part, random or anonymous chat, objectification of real people like hot or not voting, making physical threats or bulleting do not belong in the app store and may be removed without notice. So the piece piece that they added here was the random or anonymous chat.

I was gonna say because there's no pornographic content on ⁓ in apps on the app store ⁓ allegedly. I guess I think y you could you could argue that maybe Grok was ⁓ was ⁓ edging towards that, ⁓ or inching towards that. ⁓ the but to my knowledge that there hasn't been apps so that that's already been done, but the random or anonymous chat. So there there must have been some apps that have been submitted about trying to get anonymous chat with people or something. They had to put that in there.

⁓ see. Yeah, ⁓ I I haven't used any anonymous or random chat, but ⁓ anyway, for whatever reason for whatever reason they added that phrase random or anonymous chat. So if you're thinking about making a random app that a random chat app that just hooks people up with someone random to talk to, probably don't. Waste your time. There you go. ⁓ so that's helpful news of the day. AI is taking all the news and all the RAM, and we're not gonna talk about it because it's very annoying. ⁓ right.

No. It is hard to avoid. It's hard to avoid. I did discover something that's not AI related, ⁓ that did dazzle me.

Bum, bum, bum, bum. You need we need to have a big flashy intro. I assume you're gonna one of us is gonna edit this, and then it's gonna be Dan's Dazzling Discoveries. And this week Dan has discovered.

My dazzling discovery of this week is a recent video by Flow Writes Code. ⁓ he's a YouTuber, ⁓ blogger, Flow Florian, maybe. No, I think it's just Flow. ⁓ Flow Writes Code is his ⁓ handle on X. And ⁓ and he has a he does these nice videos, they're fairly short. He's I I recently saw one that was just talking about animating SF symbols, and it caught my attention because it it's like low-hanging fruit for adding subtle animations into your app. If you're using SF symbols at all, we'll we'll put a link in the description ⁓ to the YouTube. Yeah, they're super easy to use.

But you probably are. And if you're not, you really should be, because there's there's so many of them. They're very, very convenient. ⁓ the only thing you would think of as a downside about them is that if you do a multi platform app, you only have them on Apple. So ⁓ I don't care about that. So but they are so many options, so many options. And go ahead.

Yeah, I No, I I don't either. I l I Yeah, I love SF symbols. They're they're very easy to use, but I don't animate them. And and I don't like animations a lot because you I think you can overdo it. But these are very subtle little animations and you can generally trigger these animations with a one line animation modifier on the view and I won't go into it because you can watch Flo's video link in the description but think about animating your SF symbols because ⁓ they are little cute little animations that are very subtle and they add just maybe that little cherry on top of your app that ⁓ that would set it apart a little bit.

Okay. Well that's cool. And so the video shows you how to do it and I assume.

Yeah, yeah, yeah, it's it's it's very good. So what's the big item of the day though, Drew?

Right, excellent. ⁓ let's talk about app accessibility. I think that is the big item of the year, actually. ⁓ the year. And I'll tell you why I think that. ⁓ you know, accessibility we always think of as being the thing for, you know, helping the people that have various disabilities, like, ⁓ can't see very well, you can make your font big, or you know, maybe you can have a thing that reads the screen. And you know, we've seen the Apple demos on their for the on their videos where they're always do these really wonderful demos that are kind of amazing and you're like, ⁓ my gosh, this guy can navigate this.

The year, okay.

Through the apps and click on things and all this stuff. Well, ⁓ Apple ⁓ you know, has definitely been express ⁓ has always been a strong accessibility supporter, I would say, ⁓ enthusiast or whatever. But they keep mentioning it again and again, and and I think this is why probably other people thought about this, but ⁓ accessibility is well, first of all, it's a thing everyone needs eventually or at some point in their life, and

Right.

So it's not just for other people, it's probably for you. and and ⁓ some of

One of the things that I heard at a ⁓ at a at a I think it was at one of the deep dish conferences, they talked about how you maybe you don't feel like you have accessibility needs today, but let's say you get into a car accident or you break a hand or you ⁓ or something inhibits your ability to ⁓ to to interact with something normally, in whether that's you know, vision or or you're in a loud environment, you can't hear your phone, or or you can't ⁓ see something. So you need these ⁓ features to be able to interact in

Right.

Ways that may be temporary, and while we only generally think about accessibility as a permanent disability for people, it's not actually the case.

Okay. Yeah. Well that's a good point. but ⁓ but the the angle I'm gonna take on it is a little bit different. ⁓ last year, I think it was at WWDC, they talked about one of the things they mentioned is that they added these accessibility nutrition labels to the app store. ⁓ so every app you can look at its accessibility nutrition label and it'll tell you in I don't know, different categories, you know, does the app support this or support that? And you know, they encouraged you to make your app accessible and to Update your app nutrition labels, et cetera. ⁓ I haven't updated any of mine and I know a couple of my apps are not super accessible, at least in a couple ways. And I know ⁓ one thing I ran into with the client app was they had a lot of places where they were not using buttons. They were instead using, let's say, a label that you could click on or an image that that had a, you know, like a click handler thing or a tab handler. And those things are not buttons according to the what the accessibility system ⁓ gives you. If you've never used it, if if you go into your Xcode and click on what is it? Is it? I think you click Xcode up in the corner and come down to What does it say? Tools or something? Developer tools? Anyway, there's a thing under there that is ⁓ accessibility inspector. I just tried to open mine, but it wants me to update to a new version of EC code. So I will not tell you that exact path right now. But there's a thing called accessibility inspector that you can click on any window open on your Mac or go into your simulator and see get it, you know, you can see what it sees like what's available there. Like you can see, ⁓ There's a text item that says blueberries and there's a button that says okay and one that says cancel ⁓

This is an accessibility inspector within Xcode.

With it ⁓ it's ⁓ comes with Xcode, it's actually a separate app, but if you go okay, here it is. If you click on Xcode menu up on the top menu bar, go down to open developer tool, you have like instruments, simulator, icon composer, reality composer pro and then accessibility inspector. And if you click on that, could choose ⁓ one app you want to click on. It'll say select element to inspect, and you select one like you click on

Yes, yes.

Safari or one app that you're working on, and it'll give you this sort of view of what what hierarchy of the of the UI is that can be seen through accessibility. ⁓ And I was about this recently because here's the spoiler. ⁓ accessibility I think is gonna be the biggest thing this year for sure at WWDC. And I think that because ⁓ accessibility is the easiest, best path. to having AIs, AI things control apps. I've got a little like a little AI agent project. I'm not going to talk about right now because it's boring. But one of the things I was going to do is I was going to give it access to control things on my Mac. And the way I was going to do that was through this accessibility APIs. I didn't really a hundred percent what is available through accessibility APIs. And that's why I made this little and I'll put a link in here Mac OS Accessibility Client, it uses the accessibility APIs to look at various parts of your screen. Kind of the same thing as Apple's accessibility inspector, but it looks, it does it a little bit different way. And I wanted to understand how that works in the code and what things you can see, what things you can't see. Good news, you can see in simulator, you can see all the iOS stuff in there also. So if you run your app in the simulator, you can go into the simulator and say, ⁓ yep, that's a button, that's an image, this is clickable, this is not. ⁓ and you can see which things are clickable, which things are not clickable. What things you can press on, you can scroll, scroll views. ⁓ and you can and you can drive, you can control these things, you can drive them with accessibility. So like you can go and say, Click on this, scroll down to that, click on this, click on this, etc. ⁓ and yeah.

So now going back to that client app you were talking about with the with the label or that was an image with a with a on tap gesture recognizer, I assume is what it had. So that would show up ⁓ in accessibility as n as something different, or would you be able to tap it?

Yeah. Yeah, y you would see it as well if it was an image, it'll show up that this was like an image or and you could still say that you want to tap on it or press on it or whatever. But but a lot of times the accessibility app, like the the thing that's trying to drive your user interface, like you know, in a in the traditional sense way that would be, you know, some kind of accessibility thing, like maybe it's a special device that you know, looks at your UI and gives you things that you can use with buttons or maybe some of the built-in things, but in in the sort of AI world, think of it as like the AI agent or the AI LLM. ⁓ you can give it tools to use and it'll use the tools. And, you know, tools might be very well be something like what's on the screen right now. Okay. I, you know, have these things and here's the hierarchy roughly, and you know, here's the ones that are buttons. Here's the ones that things can scroll. yeah, you can click on images, but ⁓ You know, you can click on images, you can click on labels, text labels, click on anything. And I probably should have had a nice little video ready for you, but

The point being if Yeah, but the the point being if if if your app is already built with accessibility in mind or you've gone through the trouble of of putting these accessibility labels and modifiers in the views and on the buttons and on the images and and the text, your ability to adapt to what we're predicting here, th is this a WWDC prediction? Because these are bold statements.

⁓ I it it is a bold statement, and I believe, yes, it is my biggest WWDC prediction, which I've never made WWDC predictions, but yeah, my prediction would be that this will be the number one headlining thing at WWDC, will be accessibility related. And because of AI, because you're gonna be able to use it to drive AI, because like, you know, Apple talked about this a long time ago, I think. I feel like they said something like they wanted to have this thing where app where you know the AI could like kind of

Because of the ability to use an AI agent to interact with your Sure.

see your whole device and know what's going on and help you at different points. the easiest way to do that would be, you know, you've got like App Intents where you can declare ⁓ declare to your to your app like explicitly, ⁓ but you've got sort of the more generic accessibility thing where it can see buttons and click things and just act like a user would do. Now the App Intent is maybe a more direct way

Yeah, I was gonna say the

And and and I would direct I would say probably supporting that. That was what I actually had in my mind, ⁓ that I would probably should have been working on this last year is making any of the apps I do have ⁓ so that everything they could expose via app attense is exposed via app attense because then you could hopefully later say, Hey Siri, have my app do this thing and it would just do it, or I want to do this thing and it would know that the app can do it. And I think that's absolutely key. But I also think having the AI being able to drive your app is gonna be, you know, either number one or number two. And

Yeah.

And

I think it'd be big. Yeah. was gonna ask and not to put you on the spot, but but what is the difference between App Intents and accessibility labels? From a from a ⁓

⁓ you are putting me on the spot then. ⁓ Okay, so accessibility labels are things like let's say you have a button and you know it's a button because it's it's a ⁓ it's button type. ⁓ And know, but ⁓ maybe label is I'm trying to think, maybe the text on the button, or maybe the button isn't got a text, it's not ⁓ just a text button. Maybe it's like a button where you put an image on there, or maybe you drew ⁓ some kind of thing or have an image and another little thing. And it might not be like what the point of it is. Like

Could there be an SF symbol on the button?

There could be an SF symbol on the button. And I think the SF symbols are helpful that way. But like, for example, if you used one that's like a square, right? ⁓ that could be a stop button. But it doesn't say the word stop. And so that's a thing where you could put an accessibility label on and saying stop button. You know, you might put the word stop with the label with the square. You know, maybe the UI user or the user may might only see the square, but there would be this sort of hidden underneath. There's like a little piece of data.

Mm-hmm. Got it.

with that button that says, ⁓ this button is a stop button. ⁓ to stop whatever you're doing, playback or something. ⁓ so that's like one case where you might use it. And you know the case I mentioned earlier was like, you know, if you have like a like a label or an image that's clickable because you used a let's say an on tap is it on tap gesture. On tap gesture in Swift UI. On tap gesture in Swift UI isn't the same as having a button. And I recommend you try not to use that if you can use a button.

Got it. Okay. ⁓ yes, yes. Yes. Now that let's go back to that button with the square. So if we put an App Intent on that button, is it

⁓ So no App Intent you ⁓ you basically declare things your app can do. Like Let's say you have a app that files your taxes and you can say, Yeah, it's something real easy and you could and you could have like a you might have an App Intent to

Okay. Something easy.

you know, include a note to yourself for next year's taxes. You know, so so for example, there might be an interface you might go on the app and be able to say, ⁓ include a include a note, you know, add a note for next year. Okay, great. Click add a note, type in something, hit save. That's the normal way. The the App Intent think of that as anything you can do with

Okay.

What's the ⁓ automation thing we can use on our phone? What's it called? Damn. Short shortcuts. Like shortcuts. Yeah. Like there might you might be able to have a thing. If you if you set up an App Intent to say, okay, declare a, you know, add a note for next year's taxes, let's say, or what, or something like that. And it takes an input of a string, which is the the note text. Maybe that's all it takes. It's just that one thing. And it gives you no output back, but it could give you output back. ⁓ but you declare that in ⁓ in a ⁓ P list.

⁓ like a quick ⁓ what's that called shortcuts, thank you. Yeah. I never use it.

I think it's you still do it in a P list, maybe. I don't know. I'm probably wrong on that. You declare it anyway in your app, and it's part of your app bundle and it's like sort of metadata that goes along with it. So this operating system knows that, you know, Dan's ⁓ tax advisor app That it has this ability that you can use from the outside called add a add a note. And so then you could do things like put it in your shortcut to say, hey, ⁓ whenever you know you have a thing in your shortcuts so that you can run and you can tell Siri, hey, add a note to my tax advisor app for taxes next year, you know. But you know, you could easily also say if you had an AI like a smarter than the current Siri, AI, you could say, hey.

I see. I see.

⁓ AI thing, add a note to my tax thing. Any anytime I'm talking about ⁓ you know my business and money, and I might say add a note, just add it to my thing through this tax app, you know. And it would know might know how to do that because that would be an exposed feature. It's like, okay, hold on, what's that tax app? yeah, there's the app. Found the app. Okay, found the add a thing. Great. Add a note. So now it knows how to do that. And that would be exposed to the to the LLM as a tool. Yeah, like a tool call.

So this is more the the the ⁓ App Intents is is the app as a whole, whereas is accessibility is you're getting more granular. ⁓ yeah, yep.

Very granular. Yeah. Accessibility is yeah, is met is very granular and it's not really intended as like ⁓ I mean it's kind of intended as an interface for other software to use to make things accessible to you. You know, whether that's the operating system in, you know, having voice over things that it can read, or if it's ⁓ you know, things where another app is controlling it in some way, but because it's giving you access to some external device that's helping you, ⁓ you know.

Now looking ahead ⁓ for WWDC or or looking to today. ⁓ go ahead. Okay. Okay.

So, okay, wait, hold on one second. I want to reverse two things I said. Because I said that I expected the the number one thing to be the accessibility ⁓ labeling, and but I think the number one is probably going to be expanded App Intents and the number two is gonna be the the nutrition stuff. I do think that ⁓ you know, spending some time to go through your app, check it's successive how it shows up in the accessibility things, ⁓ and then do the accessibility labeling in App Store Connect, ⁓ the accessibility nutrition label in App Store Connect, I think will serve you well come fall. ⁓ I think Apple's gonna be, you know, like they always like to do, they're gonna be featuring their their hot new thing, which is ⁓ you know, I don't I I don't want to talk too much about AI, but I think it's probably gonna be AI related again. And it's gonna be, you know, hey, look at the things you can do now. And you can use any of the LLMs you want to use, maybe. I don't know what they're gonna do really, but ⁓ but the point is I think they're gonna I think your if your things are do expose a bunch of app intents and have the good nutrition labeling, that's gonna give you ⁓ that's the place you need to be, ⁓ in the app store. Yeah, I think so.

You're gonna have a head start. I I I admit ⁓ I had to go and look and and see how or or if I had gone in and done the nutrition label on my app and I had not. So ⁓ and and it has not been a a major feature ⁓ a thing that I do is to go through and label everything very accessible and test for accessibility. I I do rely on basically just the built in Swift UI ⁓ you know, framework and buttons and and labels and everything to to have the you know, some accessibility kind of built in, but I know that I am guilty of not embracing accessibility like some do. ⁓ the the person that comes to mind the most in the community is Chris Wu. I know he is very big on accessibility. Chris Wu, yeah. ⁓ he's ⁓ he's ⁓ part of the community kit organizer. He organizes ⁓ the iOS dev happy hour which I never participate in or I I have, but it it's just it's usually on a Saturday and I

Sorry, I still got my cold.

Yeah, no worries. ⁓ it it he he's he's definitely a a a prominent member of the community and he's very big on ⁓ accessibility ⁓ and and I just admit that I haven't taken the time to go through and do it. ⁓ b but, you know, this might be the time to do it because ⁓ like you say, ⁓ if if the prediction is anywhere correct, the nutrition label, the accessibility on the on

Yeah, and and I had thought about I you know, last you know, whatever, nine months ago, whatever sometime in the last year, had thought about the whole app thing and feeling like, ⁓ I really have to do those. ⁓ I to get those, I have to get my widgets made, I have got a lot of things I need to do to to make ⁓ the apps a little more you know, and the live activities, ⁓ all that of stuff kinda goes together to me. And it's you know, the better integration you have with the platform, the more people that like that platform will prefer your app. ⁓ And I and it was only really recently I started thinking about the ⁓ the accessibility width as that's a big piece of that as well. ⁓ I I think if this thing will let me share the screen, I'm gonna show you one other thing related here. Let's see. Screen. So ⁓ we're getting ⁓ this was I don't know, a week or two ago. We got this email that was brought us to this that says, hello developer, back in Earlier part of May. Recognizing impact of those. So what is this? This is saying, you know, we love our developers. And ⁓ look at these cool stickers that you don't have. And I mean, it says, like, yeah, that's great. Send me the stickers in the mail and then I'll be happier. Okay. But then ⁓ you keep going down and it's like, whoa, what is this? Prepare your app for accessibility nutrition labels. Learn how to make the most out of accessibility nutrition labels by supporting essential accessibility features. Discover how you can enhance interactions.

new WWC twenty six app. Prepared for accessibility and nutrition labels.

And this one I think came out, is there a date on this? Let's see. I don't know. It says May 2026. It was probably like maybe May 15th, roughly. So what is that? Three weeks before WWDC? Just kind of a little reminder. Hey guys, accessibility and nutrition labels. I don't know what else they had in here, but it is kind of a hint, right? But like, yeah, Apple just decided to put that that recently and like, ⁓ my gosh. So that just makes me feel again like I'm probably pretty far behind on on that stuff.

Maybe Right. Yeah, it's a hint. It's a hint. ⁓ Right.

Unshare my screen. Hopefully that was it and I didn't just wreck everything. Okay. yeah. So what do we need to do for all that? ⁓ you ask.

There you go. Good work. What do we need to do?

I don't know, you probably need to watch the videos, honestly. I think ⁓ honestly, ⁓ you know, with with the ⁓ accessibility inspector ⁓ or the little client app I made, I mean the ⁓ the way that works is you know you can you know you get a little list of like the processes from the client side, right? Not from in your app, but it'll be able to get it get a list of you know your processes, what

⁓ okay.

Each one is got on there and it can just do things like send clicks. I also ⁓ was checking ⁓ it doesn't matter. The other thing's unrelated. I was gonna I put it on Mac OS. I figured out some other stuff, but it's kind of not relevant. I think for your app, the main thing you have to do is go with accessibility inspector or go download my little test app. It's on my GitHub, Mac OS dash accessibility dash client, and I will put a link. ⁓ but it makes it really easy to see what the outside things can see about your app. ⁓

Okay.

⁓ no, it ⁓ it kind of does ⁓ on well let me back up. The accessibility inspector that comes with Xcode. ⁓ it will let you like you can you can check a map ⁓ Mac app. You can also say on the simulator, I want to do the simulator and and then choose something on there. Or you can say, I think you can even do a connected phone or something. ⁓ I haven't tried that, but I think that's a a a choice. on

Okay, nice.

My little app it I've made it from Mac and but if I select the simulator app, there's like a little sub bundle under the simulator that is the That is all the stuff of the whatever's running in the simulator. So like say the iOS simulated device, ⁓ and it's the same kind of thing. ⁓ But what you want to do is you want to see there and you want to make sure that, you know, if if I've got a clickable thing or a scrollable thing, or maybe you have a swipable thing because you've done your own gesture things, or ⁓ you know, if if ⁓ if if it's hard when you start looking at it to figure out what the you know, what you would need to activate or click or how to do that gesture. ⁓ it's gonna be hard for certainly hard for the AI, certainly hard for any other accessibility thing. ⁓ I think that's one half and I think the other half is the the app intents. which is figuring out what your app can ⁓ expose to the operating system that will make it ⁓ useful for your users who want to have a way to interact with it.

okay, thinking about to-do list now going forward, Taking everything in From what you've said, I I I think on my to-do list now between today and WWDC and moving forward, like to go through the app and see how accessible it is using using your tool or Xcode's tools or even voiceover. I I know that that's a tool, just just turning on voiceover on your app and interacting with it. ⁓ make sure you know how to get out of voiceover is a trick. ⁓ so ⁓ but I I think the prediction is solid that that there

Right. Yeah. Try to use it.

little hints, ⁓ leaving little breadcrumbs for us going forward ⁓ into WWDC, the importance of those ⁓ accessibility nutrition labels ⁓ is ⁓ it's I I think you're I think you're really onto something. I I had not put those two pieces together, but ⁓ it's a good point.

Right. Yeah. ⁓ I think that's all we got this this week. ⁓ anything else, Dan? All right. Out of time, out of out of time. So all right. Well thank you again, everyone. ⁓ we'll talk to you again soon.

We're out of time. No, it's been great. This is Thank you.