Looks like the Home application breaks non-trivial predicates?

I've just bumped into a weird problem. I've created a pretty simple and perfectly working event trigger with a non-trivial predicate which is just an AND of two characteristic-value ones.

Then I've renamed it in the Home application, and it stopped working. A dump suggests the cause is a broken predicate, for it looks now rather differently — one of those two characteristic-value predicates seems broken (details below).

Is that possible? Could please someone check whether the problem is really a Home ugly bug, or whether I do something wrong at my side? Versions are

  • macOS 27.0.1 (26A434)
  • Home 11.0 (1241.1.7.4.3)

I would gladly post a bugreport, but at this moment I am far from being sure I do not overlook something of importance myself. Thanks!

My code — for clarity stripped of all the error checking and reporting — looks like this:

    HMHome *home=...;
    HMCharacteristic *knihovna=..., *stul=..., *sklenik=...; // light chars, found by names
    HMActionSet *switchOff=...; // trivial switch-off for another light, found by name
    HMEventTrigger *trigger=...; // found by name if exists

    if (trigger) { // if found, the predicate is logged out and that's all
        NSLog(@"=== predicate found: %@", trigger.predicate);
        return;
    }
    // otherwise, the trigger and predicate are created this way:
    HMCharacteristicEvent *evt=[[HMCharacteristicEvent alloc] initWithCharacteristic:sklenik triggerValue:@1];
    NSPredicate *p1=[HMEventTrigger predicateForEvaluatingTriggerWithCharacteristic:knihovna relatedBy:NSEqualToPredicateOperatorType toValue:@1];
    NSPredicate *p2=[HMEventTrigger predicateForEvaluatingTriggerWithCharacteristic:stul relatedBy:NSEqualToPredicateOperatorType toValue:@1];
    NSPredicate *p=[[NSCompoundPredicate alloc] initWithType:NSAndPredicateType subpredicates:@[p1, p2]];
    NSLog(@"=== predicate created: %@", p);
    HMEventTrigger *et=[[HMEventTrigger alloc] initWithName:@"TEST-ONLY" events:@[evt] predicate:p];
    [home addTrigger:et completionHandler:^(NSError *error) {
        if (!error) [et addActionSet:switchOff completionHandler:^(NSError *error) {
            if (!error) [et enable:YES completionHandler:^(NSError *error) {
                if (!error) NSLog(@"=== success!");
            }];
        }];
    }];

As already mentioned, the trigger works perfectly (is trivial enough, would be weird if not). When created and also later, when found by name, the predicate looks like this:

=== predicate found: (characteristic == <HMCharacteristic, Unique ID: 7444CCCE-1C1B-5234-8CAE-01DA4D811B8B, Type: 00000025, Instance ID: 9> AND characteristicValue == 1) AND (characteristic == <HMCharacteristic, Unique ID: 2E046807-9127-565F-B9CB-10BC0F2EE6D2, Type: 00000025, Instance ID: 9> AND characteristicValue == 1)

Then, I just go to the Home application and rename the trigger, nothing else. It immediately stops working properly, and from that moment on, the predicate looks like this — note the difference:

=== predicate found: characteristic == <HMCharacteristic, Unique ID: 2E046807-9127-565F-B9CB-10BC0F2EE6D2, Type: 00000025, Instance ID: 9> AND (characteristic == <HMCharacteristic, Unique ID: 7444CCCE-1C1B-5234-8CAE-01DA4D811B8B, Type: 00000025, Instance ID: 9> AND characteristicValue == 1)
Answered by DTS Engineer in 915772022

I would gladly post a bug report, but at this moment I am far from being sure I do not overlook something of importance myself. Thanks!

So, in general, it's better to file a bug report than not. Even if there is some mistake on your part, there's value in simply documenting the fact that you had trouble with the API.

However, in this case:

Is that possible? Could someone please check whether the problem is really a Home ugly bug?

Yes, this is very likely to be a problem with Home.app. I don't know what the specific issue might be, but architecturally, the HomeKit API is capable of creating predicates that are much more complex than Home.app is capable of displaying in its interface. To a large extent, this is a side effect of HomeKit's development history— the original predicate API was created as an abstract, flexible API that tried to give developers as much power as possible.

The problem with that approach is that ACTUALLY creating a viewer/editor that's capable of displaying everything that's possible is HIDEOUSLY complicated and, in fact, I'm not sure any app has actually written one. In practice, that means that most (likely "all") HomeKit apps that interact with the predicate system have predicate configurations they can't actually create or display.

As originally envisioned, this wasn't necessarily a problem— the expectation (and reality) was that most users would pick a primary controller app they "liked" and use that to create whatever automations they wanted, which means they'd never notice that there are configurations that particular app couldn't handle.

The problem then becomes "What should a controller do when presented with a configuration it can't represent?“— leave it alone or alter the predicate to something it can represent? The standard answer is to leave it alone; however, that means you're basically "isolating" automations where the predicate can't be modified. That might sound unlikely, but keep in mind that the shared home architecture means users can create automations for "each other.” So it's entirely possible for one user to create an automation that another user couldn't display/edit without paying money for an entirely new app.

Now, one thing to understand here is that all of this is true for any controller app, not just Home.app. All Home.app really changes here is that it’s become the controller app used by the vast majority of users.

In any case, the last time I looked at this, Home.app avoided modifying predicates; however, the problem here is this:

Then I've renamed it in the Home application, and it stopped working.

I'd have to look at the code, but it's very likely that HomeKit is doing a save of the entire automation, not just the name, which could mean that it then overwrote the previous predicate with “its own.” Arguably that's a bug; however, the downside of preserving automations is that you then need additional UI so that the user can "reclaim” a predicate Home.app doesn't recognize. That's not impossible to imagine; however, Home.app has generally done everything it can to minimize complexity, and that's significant extra complexity for functionality that isn't in very wide use.

or whether I do something wrong at my side?

Not in terms of code. I thought I said this in a different thread, but my main advice is to avoid modifying complex automation outside of the app that created them.

__
Kevin Elliott
DTS Engineer, CoreOS/Hardware

Accepted Answer

I would gladly post a bug report, but at this moment I am far from being sure I do not overlook something of importance myself. Thanks!

So, in general, it's better to file a bug report than not. Even if there is some mistake on your part, there's value in simply documenting the fact that you had trouble with the API.

However, in this case:

Is that possible? Could someone please check whether the problem is really a Home ugly bug?

Yes, this is very likely to be a problem with Home.app. I don't know what the specific issue might be, but architecturally, the HomeKit API is capable of creating predicates that are much more complex than Home.app is capable of displaying in its interface. To a large extent, this is a side effect of HomeKit's development history— the original predicate API was created as an abstract, flexible API that tried to give developers as much power as possible.

The problem with that approach is that ACTUALLY creating a viewer/editor that's capable of displaying everything that's possible is HIDEOUSLY complicated and, in fact, I'm not sure any app has actually written one. In practice, that means that most (likely "all") HomeKit apps that interact with the predicate system have predicate configurations they can't actually create or display.

As originally envisioned, this wasn't necessarily a problem— the expectation (and reality) was that most users would pick a primary controller app they "liked" and use that to create whatever automations they wanted, which means they'd never notice that there are configurations that particular app couldn't handle.

The problem then becomes "What should a controller do when presented with a configuration it can't represent?“— leave it alone or alter the predicate to something it can represent? The standard answer is to leave it alone; however, that means you're basically "isolating" automations where the predicate can't be modified. That might sound unlikely, but keep in mind that the shared home architecture means users can create automations for "each other.” So it's entirely possible for one user to create an automation that another user couldn't display/edit without paying money for an entirely new app.

Now, one thing to understand here is that all of this is true for any controller app, not just Home.app. All Home.app really changes here is that it’s become the controller app used by the vast majority of users.

In any case, the last time I looked at this, Home.app avoided modifying predicates; however, the problem here is this:

Then I've renamed it in the Home application, and it stopped working.

I'd have to look at the code, but it's very likely that HomeKit is doing a save of the entire automation, not just the name, which could mean that it then overwrote the previous predicate with “its own.” Arguably that's a bug; however, the downside of preserving automations is that you then need additional UI so that the user can "reclaim” a predicate Home.app doesn't recognize. That's not impossible to imagine; however, Home.app has generally done everything it can to minimize complexity, and that's significant extra complexity for functionality that isn't in very wide use.

or whether I do something wrong at my side?

Not in terms of code. I thought I said this in a different thread, but my main advice is to avoid modifying complex automation outside of the app that created them.

__
Kevin Elliott
DTS Engineer, CoreOS/Hardware

As always, thanks a lot for a detailed insight!

When I have more time, I shall try to create a reasonable bug-report.

As for

to avoid modifying complex automation outside of the app that created them

absolutely, I do heed that advice. Nevertheless I did not modify anything in the slightest; I've just renamed it. Consider if you renamed a file in Finder and it changed its contents anyhow!

In this case, Home very definitely should do nothing else than -[HMTrigger updateName:completionHandler:].

As for

it's entirely possible for one user to create an automation that another user couldn't display/edit without paying money for an entirely new app

quite, that's a reasonable approach, provided
(a) it happens only upon edit, not just renaming (or ena/disabling)[1];
(b) it is accompanied with a warning dialogue roughly of kind “This automation contains parts unsupported by Home; do you want to edit it anyway (in which case the result might differ from the original)? [Yes][No]“

[1] Edit: due to the HomeKit API structure, the trigger contents should be also completely impervious to adding/removing scenes (possibly but for trigger-owned ones, whose API is private — why on earth?!? — and thus unknown to me).

In this case, Home very definitely should do nothing else than -[HMTrigger updateName:completionHandler:].

I'm merely describing what's going on and why, not claiming it worked well.

[1] Edit: due to the HomeKit API structure, the trigger contents should be also completely impervious to adding/removing scenes (possibly but for trigger-owned ones, whose API is private — why on earth?!? — and thus unknown to me).

So, the basic situation here is a result of a very complicated release history. Basically:

  1. We built an abstract framework that seemed "reasonable".

  2. Two years later, we built an app with that framework and quickly discovered that the abstract framework's design didn't really match how a usable application should actually function. This is particularly true of HMActionSet, which only vaguely resembles how Home.app's UI actually works.

  3. Since then, we've added functionality (notably, Shortcuts support) which isn't exposed in the public API and which would be extraordinarily difficult to add support for.

The final factor here is that the cross-app compatibility HomeKit was designed to support has, in practice, just not happened, at least not when it comes to "general controller" apps. That is, the vast majority of users' usage patterns look something like this:

  1. They have ONE "general controller" app. In most cases, this is Home.app, but it could be some other general controller app. However, the critical factor is that ONE app is what they actually use for things like automation management.

  2. They have some number of vendor-specific apps that provide functionality for specific accessories (firmware updates, managing special features, etc.). These apps have minimal (if any) support for other accessories and don't really interact with the broader home.

  3. (optional) They MIGHT have one or more "special-purpose" apps (like dashboard apps) which, again, don't really modify the broader home.

...which basically means that maintaining HMActionSet compatibility hasn't really mattered. I can certainly understand why this is frustrating to deal with as a developer, but the dynamics above explain why, in practice, it hasn't really mattered that much to our users, which is what we're primarily concerned about.

__
Kevin Elliott
DTS Engineer, CoreOS/Hardware

Thanks again for further insight, and sorry for the “why on earth?!?” part; I just was letting out some steam from the frustration that such a nicely designed and potentially highly useable and handy API[2] does not work by far as well as it could and should :(

Is there, at the very least, somewhere a reasonably complete and reliable documentation, even if a form of developer notes or something like that?

Lately I've bumped into (and lost non-trivial time over) the problem that predicateForEvaluatingTriggerOccurringOnDateWithComponents: seems to support hours and minutes only. I've particularly needed to limit a trigger to a couple of specific dates (something like “just on Christmas and the 4th of July“), and found the hard way it does not work, sigh. I can't see any mention of such a limitation neither here nor there [1] :( Also, frankly, I can't see any reason why such a limitation should be added; seems to me completely gratuitous, for checking simply all the non-empty DC fields one-by-other against the current date is a programmer's work for about ten minutes (nor it adds any heavy computational complexity, especially on a device which is able to interpret as complex things as shortcuts).

[1] Unlike e.g., recurrence, whose limitation actually happens to be documented all right.

[2] Incidentally — not that it is of any importance, of course — the existence and (then perceived) flexibility of that API was one of main reasons I've chosen HomeKit years ago for my home automations: other options either have no API at all, or at best they are programmed in a by-far not as-nice API and in a by-far uglier language that the nice and convenient Objective C.

Thanks!

Thanks again for the further insight, and sorry for the “why on earth?!?” part; I was just letting off some steam from the frustration that such a nicely designed, potentially highly usable, and handy API does not work nearly as well as it could and should. :(

Is there, at the very least, somewhere reasonably complete and reliable documentation, even if in the form of developer notes or something like that?

No, not really, particularly when it comes to predicate issues. To be honest, these APIs just aren't that widely used, so I get very few questions (< 1/year) about them. That means I haven't spent that much time digging into their exact behavior and “quirks,” much less putting that into a formal document.

Lately I've bumped into (and lost non-trivial time over) the problem that predicateForEvaluatingTriggerOccurringOnDateWithComponents: seems to support hours and minutes only.

Have you looked at HMCalendarEvent?

As for:

seems to support hours and minutes only.

I've particularly needed to limit a trigger to a couple of specific dates (something like “just on Christmas and the 4th of July”), and found the hard way it does not work, sigh.

I haven't tested it, but I believe this would work:

// Christmas: 25 December at 08:00
NSDateComponents *christmas = [[NSDateComponents alloc] init];
christmas.month  = 12;
christmas.day    = 25;
christmas.hour   = 8;
christmas.minute = 0;

// Independence Day: 4 July at 08:00
NSDateComponents *july4 = [[NSDateComponents alloc] init];
july4.month  = 7;
july4.day    = 4;
july4.hour   = 8;
july4.minute = 0;

HMCalendarEvent *christmasEvent = [[HMCalendarEvent alloc] initWithFireDateComponents:christmas];
HMCalendarEvent *july4Event     = [[HMCalendarEvent alloc] initWithFireDateComponents:july4];

HMEventTrigger *trigger = [[HMEventTrigger alloc] initWithName:@"Christmas and 4th of July"
                                                        events:@[christmasEvent, july4Event]
                                                     endEvents:nil
                                                   recurrences:nil  
                                                     predicate:nil];

Also, frankly, I can't see any reason why such a limitation should be added; it seems to me completely gratuitous, as simply checking all the non-empty date component fields one by one against the current date is about ten minutes of work for a programmer...

The problem is that as you widen the time from a “particular time” (like “every hour”) to a “particular day” (like “every Friday” or “June 19th”), it becomes increasingly hard to get consistent behavior across every possible edge case. For example, what should happen if an event fires “every two hours” when daylight saving time starts and stops? Similarly, scheduling yearly events in terms of day counts means that leap years shift the day every four years.

...nor does it add any heavy computational complexity, especially on a device that is able to interpret things as complex as Shortcuts.

This is actually an issue: these predicates end up getting evaluated fairly frequently and, while an individual evaluation isn't that high, the total cost over an extended period ends up being quite high. Conceptually, HomeKit handles the firing of these events by periodically processing the entire event set, determining what will fire “in the future,” and then triggering when the time arrives. Putting that in more concrete terms: every X hours, it determines what will trigger in the next X hours, then runs the action set when that timer arrives. For an event that only fires once per year, that's a lot of predicate evaluations, each of which is much more computationally expensive than a calendar event check.

--
Kevin Elliott
DTS Engineer, CoreOS/Hardware

Thanks again! Just to quickly fix the loose ends, not that it is important anyhow :)

Have you looked at HMCalendarEvent?

Thanks, I do use those elsewhere (and so far, I haste to add, they work without a glitch). In this particular case though the problem was that these triggers happen to have more complicated events to fire on; they just should be skipped/ignored unless on one of a set of particular dates.

Meantime, I've found another solution (essentially, my HK application checks daily the current state, and changes the triggers programmatically when one of the important dates is less than a week in future; this way, recurrence can be used instead of a predicate, and the danger of my app not being able to check/fix the triggers at least once in the week is very low).

... For example, what should happen if an event fires “every two hours” when daylight saving time starts and stops? Similarly, scheduling yearly events in terms of day counts means that leap years shift the day every four years ... HomeKit handles the firing of these events by periodically processing the entire event set, determining what will fire “in the future,” and then triggering when the time arrives ...

That's why I don't speak of calendar (nor any other) events, but just of the calendar-based predicates.

Whilst the former indeed might affect a pretty complex logic of determining when to fire and when not, the latter seem to me very easy to implement, and without any potential drawback: when the trigger does fire (as determined through its events, as they are, without any potential extension), only then the current timestamp is checked against the calendar predicate. It fits all the fields, the scene is run (and end events are activated). It does not fit one of the fields, the scene is not run (and end events are ignored). Nothing else needed; especially there is absolutely no need to check the predicate anyhow unless/until the event actually fires.

And here I really can't see why and how checking all the date component fields (even including those rarely used ones like the era; but at the very least also the year, month, and day) could be really more difficult or processing-heavy than checking just the hour and minute, as happens now. I am willing to bet that checking the characteristic-based predicates is both implementation- and runtime-wise very considerably harder.

Do I blindly overlook something of importance here?

Anyway, thanks again a lot!

FB25578489