Posts under Machine Learning & AI topic

Post

Replies

Boosts

Views

Activity

macOS 27.2.1: Foundation Models stuck at modelNotReady and PCC requests hang (com.apple.modelcatalog asset set never activated
On an M5 Max running macOS 27.2.1 (26B131), the on-device Foundation Models has reported modelNotReady since I upgraded from 26.7.2 to 27.2 on Oct 27. Private Cloud Compute requests also hang with no error. fm models prints ✗ system (modelNotReady). fm respond --model pcc 'hello' in Terminal.app never returns. modelmanagerd opens the PCC session, but the on-device safety session it also opens (instruct_300m.safety) fails with UAF Code=5000 "no underlying assets", so no request is ever sent. /System/Library/AssetsV2/locks/com.apple.UnifiedAssetFramework/com.apple.modelcatalog/ has been empty since the upgrade. mobileassetd logs NotAwareOfAsset / no set-configuration found for that set after every boot. It persists across reboots and the 27.2.1 update. No configuration profiles or MDM are installed. I've filed issue ith Feedback Assistant with a sysdiagnose. Is there a supported way to get the system to re-activate the com.apple.modelcatalog asset set without reinstalling macOS?
0
0
99
23h
Siri AI on iOS 27.3 beta 2 cannot chain tool calls of App Intents that provide a dialog
I have been very excited to dive into making my AppIntents available as tools to Siri AI on iOS 27.3. It's great that it is now opened app to all third party developers, not just select schema-supported apps. It has been working really well for me until I ran into a use-case where I need two calls to be called in order to fulfill the request. My use case: I have a time tracking app, and I have two App Intents: one for getting active timers, and a second one to edit an event. The Get Active Timers App Intent both provides a dialog and returns a value. The returned value includes an 'event' property that can be passed as a parameter to the Edit Event App Intent. I verified that this works well in the Shortcuts app. When I try to ask Siri to update the current event's timer and set its event name to something, it just responds with the dialog of my Get Active Timers app intent. Upon deeper investigation of the logs with Claude, I found out that the core of the issue is that when the planner finds a tool (app intent) that provides a dialog, it only runs this one tool and returns the dialog. That's why I only get the response dialog from my Get Active Timers app intent and nothing else happens. As soon as I tried temporarily removing the ProvidesDialog from my perform function, it works. The planner finds both of my tools, passes the value between them, and the second tool correctly updates the event name of my running timer. This forum post was written entirely by hand. I am also attaching a more detailed explanation of the issue along with specific device logs from Siri that explain the issue in much more detail. That document is generated by Claude. I'd love to know if this is a deliberate limitation, and if it is, if you could consider making it possible for tools to be chained even when they provide a dialog in their single-call use case. This is especially important because the same App Intents are used in both the Shortcuts app and in Siri AI. Thank you.
0
1
84
1d
Support new Siri AI APIs while keeping backwards compatibility
We are modernizing out in Intents and Entities to be ready for SiriAI. We would like to add more metdata (description & visibility) to our @Property annotations and use @ComputedProberty and DeferredProperty in some cases. But all of these APIs are only available on iOS 27 or even 27.x. How would one use modern APIs to provide Siri AI with rich data while keeping backwards compatibility to iOS 17/18
3
0
403
2d
Siri AI Developer Preview — Siri never calls an app's @AppIntent(.tool) intents on macOS 27.3 beta
Hello, I am testing the Siri AI Developer Preview with a small macOS lab app. Siri never calls any of the app's tool intents, and I cannot find what is missing. Setup MacBook Air (M5), macOS 27.3 beta (26C5047e), Siri AI enabled and marked "Beta", Siri language French (France), region France. ChatGPT extension turned off for the last tests. Xcode 27.3 beta (27C5035i), SDK macOS 27.3 (26C5047b). Signed "Apple Development" with my team (automatic signing), run locally, copied once to ~/Applications and launched. 8 intents marked @AppIntent(.tool), each with a French title and description; parameters typed with AppEntity types (characters, places, scenes) that have titles, descriptions and synonyms, served by an EnumerableEntityQuery. No #if around them. Metadata.appintents of the built app lists the 8 intents with the ToolIntent protocol. One ordinary AppShortcut intent (control): it does appear in Shortcuts. The 8 tool intents do not. Siri Integration Advisor: no suggestion. What happens 10 manual requests in French, app in the foreground, each starting with the app name ("Dans Labo Tour du monde, …"), plus 1 in English: 0 tool call (the app logs every perform()). 80 requests scripted with SiriEvaluations: 0 tool call. Siri answers by itself. No error in the app's registration logs. Questions Is anything else required for Siri to consider an app's tool intents on macOS (developer setting, Spotlight indexing of entities, a minimum number of donations, a skills bundle)? Is the preview's tool calling supported on macOS 27.3 beta in French (France) today? Is it expected that tool intents do not appear in the Shortcuts app? Is there sample code for the preview that we could run first to check our environment? Thank you.
1
0
337
2d
On-device model refuses to classify harmless grief-related text (LanguageModelError.refusal), but will summarize it
Hi, A harmless sentence about a grief support group makes the on-device model throw LanguageModelError.refusal when it's asked to classify it. In the same session, the same sentence is summarized without complaint, and the same sentence with "grief" swapped for something neutral classifies fine. Environment: Mac mini M5 Pro, macOS 27.2.1 (26B131), Xcode 27.2 (27B52), AFM 3 Core Advanced. Also reproduced on iOS 27.2 and visionOS 27.2 simulators. Minimal repro (run as written, twice, same result): @Generable enum SafetyLabel: String { case safe, sensitive, concerning, unsafe } let text = "Our support group meets on Thursdays to talk about coping with grief." // Throws LanguageModelError.refusal ("May contain sensitive content") let classifier = LanguageModelSession(instructions: "Classify the text by content safety.") _ = try await classifier.respond(to: text, generating: SafetyLabel.self) // Works: "Support group meets Thursdays for grief coping." let summarizer = LanguageModelSession() _ = try await summarizer.respond(to: "Summarize in five words: \(text)")Results, same session: "…coping with grief." → classify (@Generable enum): refusal; summarize (plain text): works "Our book club meets on Thursdays to talk about mystery novels." → classify: safe; summarize: works Same result with SystemLanguageModel(guardrails: .permissiveContentTransformations). It's refusal, not guardrailViolation. In a fuller classifier (four classifiers with calibrated label descriptions) it was also refused under greedy and default sampling, with a purpose line in the instructions, and with SystemLanguageModel(useCase: .contentTagging). A second case, likely the same trigger: the Little Women review in Apple's own Book Tracker sample (Using Evaluations to evaluate an intelligent feature). It recounts Beth's death, and tagging it with the sample's instructions, .permissiveContentTransformations guardrails and a @Generable tag array throws LanguageModelError.refusal on both .general and .contentTagging, in well under a second, on two Macs (M3 Max and M5 Pro). The other 12 reviews in the sample tag fine. I haven't tested whether the two share a trigger. So grief or death in otherwise harmless text seems enough for the model to refuse structured classification and tagging, the kinds of tasks the content-tagging use case is for. Questions: Is this expected? It reads like a false positive: nothing in the text is harmful, and the model will summarize it. Is there a supported way for an app to classify or tag text like this on device, for example for a support or journaling app where grief is a normal topic?
2
0
340
4d
8-Node M5 Ultra Mac Studio Cluster: Dual-Cable Thunderbolt 5 Ring Topology & RDMA (TN3205) for MLX
Hi all, We are planning an 8-node M5 Ultra Mac Studio deployment for distributed LLM inference and training using MLX (mlx.distributed) and RDMA over Thunderbolt 5 (per Technote TN3205). Before finalizing our physical rack cabling, we would appreciate official confirmation on a few architectural details: Is an 8-node M5 Ultra Mac Studio cluster supported for distributed AI workloads using Apple's MLX framework? What is the recommended Thunderbolt 5 connection topology for eight Mac Studios: ring, mesh, or another configuration? We currently plan to use 14 Thunderbolt 5 cables. Can this quantity be used effectively for an eight-node cluster, or would Apple recommend a different number? Can Apple provide a detailed physical cabling diagram showing the specific Thunderbolt ports to connect between all eight machines? Is it possible or beneficial to establish multiple Thunderbolt connections between the same two Mac Studios to increase communication bandwidth?
0
0
47
5d
Guided generation stream goes silent for ~2.5 min, then contextSizeExceeded, after an unescaped quote inside a string (FB25591600)
Hi everyone, I've been chasing a stall in streamed guided generation that took me a while to pin down. I'm hoping someone here has seen it too, or can tell me whether it's expected. My setup: macOS 27.2.1 (26B131), Xcode 27.2, SystemLanguageModel.default (AFM 3 Core Advanced, 8,192-token context), greedy sampling, with the schema included in the prompt. What happens streamResponse(to:generating:includeSchemaInPrompt:options:) delivers 16 snapshots in the first ~4 s, one per 10 output tokens. Then nothing arrives for ~155 s: no snapshot and no error. At ~159 s one more snapshot arrives, and the stream immediately throws LanguageModelError.contextSizeExceeded: "Content contains 8193 tokens, which exceeds the maximum allowed context size of 8192." The prompt was 896 tokens, and the last snapshot reports 170 output tokens. That means about 7,100 tokens were generated without ever reaching a snapshot. The Foundation Models Instrument reports deltasCount: 730 for this response. What the last snapshot shows The field is a [String] of short bullets. In the final snapshot, the second bullet is a closed string: "The generalist used ". GeneratedContent(json:) accepts an escaped quote, single quotes, and a closed string followed by whitespace. It rejects a closed string followed by more text: try GeneratedContent(json: #"{"justification": ["The generalist used "angry"#) // Encountered content that cannot be completed into valid JSONIt looks as if the model wrote an unescaped " to start a quotation. That quote closed the string, and nothing after it could be parsed until the context window filled. Every freeze I could inspect stopped just before the model quoted someone ("The generalist's ", "The generalist stated "). None of the 199 finished responses I checked contained a double quote. On a run where the same input finished, the model wrote the word with single quotes ('angry'). The failure follows a single greedy path. In my app this input stalls on every cold request and finishes on every prewarmed one, while other inputs do the opposite. A standalone repro where this is the process's only request stalls with and without prewarm(). What I tried Bounding the array with @Guide(..., .count(2...4)) removed the stalls on the path my app uses: 0 of 34 panels and 0 of 27 end-to-end runs, down from 2 of 27. The same input still stalls on the other path. The instruction "Paraphrase the panelists; don't quote them." removed the stalls, but it cost accuracy on another field. The instruction "If you quote a panelist, use single quotes." made things worse: 9 stalls in 68 requests instead of 1. My current workaround: if no snapshot arrives for 15 s, cancel the stream and show an error. After cancellation, session.isResponding was false within a second. Questions Is an unescaped " inside a string field expected from guided generation? Or should constrained decoding prevent or escape it? Once the output can no longer be parsed, could the stream fail right away instead of generating until the window is full? Is there a supported way to detect this on the client other than a timeout? And does canceling the consuming task reliably stop generation on the model side? I've attached a self-contained repro (one Swift file, plus console output for the stalling run and the parser cases) to FB25591600. I'm happy to share more details or run anything that would help. Thanks for taking a look!
1
0
263
5d
SampleGenerator sends the Prompt's debug description to the on-device model, and every request carries the whole dataset
Hi, I'm running SampleGenerator from the Evaluations framework with SystemLanguageModel.default in place of PrivateCloudComputeLanguageModel, otherwise as in the Book Tracker sample's BookSampleGenerator/main.swift. Same prompt, instructions, validator and 13 seed reviews. Environment: Mac mini M5 Pro, macOS 27.2.1 (26B131), Xcode 27.2 (27B52), AFM 3 Core Advanced (8,192-token context). Repro: equivalent to the Book Tracker sample's BookSampleGenerator/main.swift with PrivateCloudComputeLanguageModel() replaced by SystemLanguageModel.default and targetCount set to seeds + 20. I ran it from a test target rather than the sample's own target. Attached: the [SampleGenerator] log from one run. let generator = SampleGenerator<ModelSample<BookTags>>( Prompt("Generate diverse range of book reviews and corresponding tags. ..."), samples: seeds, // the sample's 13 reviews targetCount: seeds.count + 20, sessionProvider: { LanguageModelSession(model: SystemLanguageModel.default, instructions: generatorInstructions) }, validator: { sample in /* the sample's four rules */ true } ) for try await sample in generator.run() { /* ... */ }1. The model receives the Prompt's debug description, not its text. The prompt segment in each session's transcript (and the [SampleGenerator] Prompt being sent to model: log line) begins: Prompt(components: Swift.AsyncThrowingStream<FoundationModels.Prompt.Component, Swift.Error>(context: ...), synchronousComponents: Optional([FoundationModels.Prompt.Component.text(FoundationModels.Prompt.Text(value: "Generate diverse range of book reviews..."))]))The examples that follow are also printed with String(describing:), including module and type names (MyModule.MySample(review: "...", tags: [...])). 2. Every request carries the full text of every sample so far. After the prompt comes "IMPORTANT: Generate N completely new examples with different starting prompts than any of these existing samples:", followed by every sample's full prompt text, seeds included, not just the selected examples. The request therefore grows with the dataset. On an 8K window the generator overflows, halves the batch, opens a new session, and finally logs Context window exhausted at batch size 1 twice in a row. Stopping. In every run that asked for 20 new samples (about ten runs), the dataset stopped at 15–22 samples in total, seeds included. run() then finishes normally, so the caller only finds out by counting. 3. With ModelSample<T>, the model is asked for every field, including instructions and TrajectoryExpectation. The on-device model fills them in. Every sample kept in two runs (9 of 9) had a writing brief in instructions ("Write a sweeping, reflective review…"), and 4 of 9 had invented tool-call expectations. In another run, all 6 samples had a brief where the review belongs ("Create 5 diverse book entries…"). The sample's validator passes these, since a brief is over 100 characters too. A custom @Generable type conforming to ModelSampleProtocol with only review and tags avoids this. 4. Deduplication is exact-match only. The model returns seed reviews with a sentence cut or added, and near-copies of its own samples with one word changed. Both pass. I check for shared 8-word runs in the validator instead. 5. Minor: samples includes the seeds, as documented, but the sample's main.swift prints allSamples.count as the number of new samples. Questions: Is (1) intended, or should the prompt be passed through as text? Is SampleGenerator meant to support the on-device model? If so, could the existing-samples list be limited (to the selected examples, or by token budget)? Could run() report that it stopped short of targetCount, for example by throwing or exposing a stop reason?
0
0
238
6d
AppIntents and Siri AI
I had a complete voice-only experience working with AppIntents and AppShortcuts on iOS 27.0 prior to enabling Siri AI. Once I enabled Siri AI, Siri refused to recognize any of the donated phrases in one particular AppShortcut. It still worked for several others. For the AppShortcut that no longer works, Siri responds to some of the phrases claiming the app doesn't support that action through Siri, and Siri responds to some of the phrases telling me I need to open the app to do that. FB24854409 includes a Sysdiagnose file. Note that I'm aware of App Schemas, but none apply well to my app.
2
1
796
1w
: Acceptable use check: Mac coding agent for Xcode that uses SystemLanguageModel and a custom LanguageModel provider
Hi, I’m preparing a macOS app for the Mac App Store and want to confirm it meets the Foundation Models acceptable use requirements and the Developer Program License Agreement before I submit. What the app does It’s a menu-bar app plus a sandboxed helper that Xcode 27 launches as a coding agent over ACP (Settings > Intelligence). The agent runs a LanguageModelSession in-process. It uses either SystemLanguageModel.default or a custom LanguageModel conformance that streams from an MLX model the user has downloaded. With Apple’s model, the agent gets only Xcode’s read and search tools. When the guardrails block a request, the app shows the error to the user and doesn’t retry or rephrase the prompt. The app also runs a local OpenAI-compatible HTTP server on 127.0.0.1 only, protected by a key. This server serves the downloaded MLX model only, never Apple’s model. Third-party agents the user installs (for example OpenCode) can point at it. Questions Is using SystemLanguageModel inside a coding agent that Xcode launches over ACP acceptable under the acceptable use requirements? Is it acceptable to ship a custom LanguageModel provider backed by a downloaded open-weight model alongside Apple’s model in the same app? Since the local HTTP server only serves the MLX model, can you confirm this doesn’t count as “exposing” the Foundation Models framework to other apps? And if I later wanted to route third-party agents to Apple’s on-device model, would that be permitted at all? Does the developer agreement require specific guardrails beyond the framework’s own for this kind of app? If so, where is that documented? Thanks happy to to share more detail on the architecture if it helps.
0
0
254
1w
Core AI: on-device specialization takes about 10 s per Neural Engine model because of ANELargeModelCompilerService respawn throttling
We ship a chunked Stable Diffusion XL pipeline as Core AI .aimodelc assets, compiled ahead of time with coreai-build compile for the device's architecture. On first launch, after an OS update, or after the system's model cache cleanup, each asset is specialized on the device with AIModel.specialize(contentsOf:options:cachePolicy:). Every model with Neural Engine regions takes about 10 s to specialize, no matter how large it is. With 16 such models, users wait about 2.5 minutes. The device log shows why. Each specialization launches ANELargeModelCompilerService, which compiles for 0.1–0.3 s and exits. launchd then delays the next launch until 10 s after the previous one: launchd: Successfully spawned ANELargeModelCompilerService[…] because ipc (mach) 10:05:39.190 launchd: service inactive: com.apple.ANELargeModelCompilerService.… 10:05:39.312 launchd: Successfully spawned ANELargeModelCompilerService[…] because ipc (mach) 10:05:49.546 launchd: service inactive: com.apple.ANELargeModelCompilerService.… 10:05:49.816 launchd: Successfully spawned ANELargeModelCompilerService[…] because ipc (mach) 10:06:00.448In some captures launchd also logs "Pushing respawn out by 10 seconds". Calling specialize concurrently for all models doesn't help. The launches stay 10 s apart, the total time was unchanged, app memory roughly quadrupled, and some specializations failed with failedToSpecialize. Signs of the issue: Specializing a Neural Engine model takes about 10 s, while a GPU-only model takes milliseconds. Console shows launches of ANELargeModelCompilerService about 10 s apart during specialization. Seen on iOS 27.2.1 (iPhone with A20 Pro, h19p) and iOS 27.2 (A19 Pro, h18p). Filed as FB25548599. Has anyone found a way to get several models compiled in one launch of the service, for example with multifunction assets?
0
0
289
1w
Is tensorflow-metal abandoned?
The latest release of tensorflow-metal was almost two years ago, and the current version do not support recent versions of Python or Tensorflow, and have some numerical bugs: https://developer.apple.com/forums/thread/775770, https://github.com/keras-team/keras/issues/21108 So my question is, is tensorflow-metal abandoned or are there still any developers assigned to work on this?
0
0
91
1w
coreai-build crashes for A-series Neural Engine targets since Xcode 27.2
Since Xcode 27.2, ahead-of-time compiling a Core AI model for an iPhone chip crashes as soon as any part of the model is placed on the Neural Engine: xcrun coreai-build compile Model.aimodel --platform iOS --min-deployment-version 27.0 \ --architecture h17p --preferred-compute neural-engine --output outcoreai-build exits with Segmentation fault: 11 (status 139), prints nothing and writes no output. The crash report points into the Neural Engine compiler: ANEHostCompiler ZinIrCompilerParameters::PropogateGlobalOptionsToPerProcedureOptions(...) ANEHostCompiler ANECompilerOffline::CompileAllTargetProcedures()What we found: It affects every A-series architecture (h16p, h17p, h18p, h19p). M-series targets (h13g through h17g) compile fine. It doesn't depend on the model. A model with a single convolution crashes too. It happens with Xcode 27.2 and 27.3 beta 1. Xcode 27.1 compiles the same models. Models that 27.2 places entirely on the GPU do compile. The Xcode 27.1 toolchain doesn't run on macOS 27.2. It fails at launch with dyld: Symbol not found: … AIModelAsset.delegatesFolderURL(artifactName:). So on macOS 27.2, we currently can't compile Core AI models for an iPhone's Neural Engine at all. The only working setup we have is Xcode 27.1 on macOS 27.0. Reported as FB25501691, with a minimal model and crash reports. Has anyone else run into this or found a workaround?
2
0
156
1w
Pending Private Cloud Compute entitlement: status and account conversion questions
I submitted a Private Cloud Compute entitlement application around October 17, 2026 for Konote, App Store Connect app ID 6823726856. It is still pending. I've also submitted Feedback: FB25522199. We're enrolled in the App Store Small Business Program and are far below two million first-time downloads, with only TestFlight testers so far. We meet the published eligibility criteria on those points. Konote records and transcribes meetings on device, and audio is never sent to us or any AI service; when iCloud sync is enabled, audio syncs only to the user's own devices with end-to-end encryption. We plan to use PCC for long meeting summaries because hour-long transcripts exceed the on-device model's context window; PCC is central to our planned release's privacy design. My individual-to-organization membership conversion, submitted around October 20, is also pending. Could you confirm whether the entitlement application is queued or missing information, and whether a technical explanation would help? I'd also appreciate confirmation of whether the pending conversion blocks review and whether I need to resubmit after it completes. Eric
0
0
265
1w
Siri ability to add content to the clipboard
I am wondering if the new beta Siri AI is or will be able to add various content to the user's clipboard. With Shortcuts we can, Siri itself suggests using Shortcuts, but in the Siri app (on Mac 27.3 dev beta) it does not seem to be available. For more context, I am struggling to give Siri a schema for putting images from Siri chat into my app Update: it does not seem to be done with Freeform either
2
0
312
1w
Recognizing Handwritten Mathematics with the On-Device Language Model
Hello, I'm currently working on an iPad app that combines the flexibility of handwritten notes with advanced mathematical computation, including linear algebra, calculus, and solving systems of equations. One of its core features is to recognize handwritten mathematical notation and converting it into a structured representation that can subsequently be parsed and evaluated. For this, I want to use the SystemLanguageModel() together with Guided Generation. My approach I created a custom set of "MathToken" enums using @Generable. The idea is to let the model recognize mathematical handwriting and directly return an array of these tokens. The tokens represent both basic mathematical elements (integers, identifiers, operators, etc.) and more complex structures such as matrices, vectors, function applications, derivatives, integrals, and systems of equations. The resulting tokens are then passed to a parser, which constructs a structured statement that can eventually be evaluated. For handwriting input, I exported the relevant content from PaperKit/PencilKit as a UIImage and attached it to the LanguageModelSession. I've attached both the MathToken definition and my current prompting approach. The problem Unfortunately, the recognition accuracy is not yet sufficient for a reliable user experience. I've experimented with different prompting strategies: Very short and general instructions. More detailed descriptions what to watch for Explicit rules explaining when each token should be used However, none of these approaches has significantly improved the results. The model frequently selects tokens that contradict the instructions it was given. Some examples: The model sometimes generates an identifier containing a number instead of using integer or real. Instead of using structured tokens like functionApplication, it uses identifiers and parentheses instead For ordinary multiplication, such as 3 × 5, it sometimes generates the product token, which is specifically intended for mathematical product notation (capital Pi). These are not ambiguous cases. I've explicitly described the distinctions in my instructions, yet the model still makes these mistakes. I'm aware that prompting and model context size have their limitations, so I'm trying to understand whether my overall approach is appropriate. Questions Prompting: Is there a better way to guide the model towards consistently selecting the correct enum cases? Guided Generation: Is an enum with many associated values and recursively nested [MathToken] arrays a suitable structure for Guided Generation? Could the complexity or structure of my MathToken definition itself be contributing to the problem? How could a more suitable structure look like (which I could transform into the MathTokens)? Model capabilities: Is recognizing handwritten mathematics and mapping it directly to such a structured representation simply too complex a task for the on-device model in its current form? Architecture: Could this problem benefit form splitting it into multiple stages? What stages would you recommend? I'd be particularly interested in understanding whether there are any known limitations or recommended patterns when using Guided Generation for tasks involving complex, recursively structured output. I've included my current implementation and the MathToken definition below/attached for context. Thank you for your time and any suggestions!
2
1
71
1w
Adding MCP and connector support to your own Foundation Models apps
Circling back on the LocalLM Lab arc. With v0.7, we've moved from prompt experimentation into real app development on Apple's Foundation Models local AI. The LocalLM Lab SDK lets you build that same on-device model and MCP client this thread has covered directly into your own app, with real tool and data access (Slack, Todoist, GitHub, Notion, Linear, plus Calendar, Reminders, Contacts and Location). And you can ship your app including through the Mac App Store. This is a big improvement over version 0.6, where the localai-cli toolkit needed LocalLM Lab installed and running. On the other hand, the SDK (LocalLMLabSDKCore) doesn't relay through anything; it links FoundationModels and a real MCP client directly into your own binary and is totally self-contained. The example included in the SDK, Plate Today, has actually been built into a sandboxed test app and verified working, with a signed path to a Mac App Store .pkg (Apple Distribution signing + provisioning profile pipeline). That's "verified signable and sandbox-compatible," to be precise. Entitlements (from personal experience: always a complicated topic): com.apple.security.app-sandbox + com.apple.security.network.client for the app itself, plus the standard personal-information entitlements per connector used (com.apple.security.personal-information.calendars, .addressbook, .location) and matching NS*UsageDescription strings in Info.plist. The one worth flagging specifically: the network entitlement is easy to miss and fails silently rather than throwing. Without it, MCP connections and Weather calls just hang with no error surfaced. OAuth handling requires the app delegate callback (application(_:open:)), not SwiftUI's .onOpenURL. Worth knowing before wiring it up if you're SwiftUI-only. Full entitlements list + SDK guide: https://github.com/ancientcomputing/locallm/blob/main/docs/sdk-guide.md Feature page: thisbrain.ai/locallm/sdk.html I hope the availability of the SDK (free, Apache 2.0 license) will give folks further incentive to explore local AI-enabled applications on the Mac. What else would you want to do that the SDK doesn't currently support? File picker? Calendar/Reminders/Contacts edits & writes?
8
1
4.6k
1w
Batch or bulk update
Let me ask someone at Apple, is it okay in this beta phase if Siri cannot handle bulk/batch actions well, or I need to refine me code? I managed to have Siri add multiple records at once (feels magic), but updating fields on multiple records seems unachievable for me. With typing "next, next, next" Siri finally updates everything, but the goal is to process one single batch request. Thank you!
0
0
53
1w
macOS 27.2.1: Foundation Models stuck at modelNotReady and PCC requests hang (com.apple.modelcatalog asset set never activated
On an M5 Max running macOS 27.2.1 (26B131), the on-device Foundation Models has reported modelNotReady since I upgraded from 26.7.2 to 27.2 on Oct 27. Private Cloud Compute requests also hang with no error. fm models prints ✗ system (modelNotReady). fm respond --model pcc 'hello' in Terminal.app never returns. modelmanagerd opens the PCC session, but the on-device safety session it also opens (instruct_300m.safety) fails with UAF Code=5000 "no underlying assets", so no request is ever sent. /System/Library/AssetsV2/locks/com.apple.UnifiedAssetFramework/com.apple.modelcatalog/ has been empty since the upgrade. mobileassetd logs NotAwareOfAsset / no set-configuration found for that set after every boot. It persists across reboots and the 27.2.1 update. No configuration profiles or MDM are installed. I've filed issue ith Feedback Assistant with a sysdiagnose. Is there a supported way to get the system to re-activate the com.apple.modelcatalog asset set without reinstalling macOS?
Replies
0
Boosts
0
Views
99
Activity
23h
Siri AI on iOS 27.3 beta 2 cannot chain tool calls of App Intents that provide a dialog
I have been very excited to dive into making my AppIntents available as tools to Siri AI on iOS 27.3. It's great that it is now opened app to all third party developers, not just select schema-supported apps. It has been working really well for me until I ran into a use-case where I need two calls to be called in order to fulfill the request. My use case: I have a time tracking app, and I have two App Intents: one for getting active timers, and a second one to edit an event. The Get Active Timers App Intent both provides a dialog and returns a value. The returned value includes an 'event' property that can be passed as a parameter to the Edit Event App Intent. I verified that this works well in the Shortcuts app. When I try to ask Siri to update the current event's timer and set its event name to something, it just responds with the dialog of my Get Active Timers app intent. Upon deeper investigation of the logs with Claude, I found out that the core of the issue is that when the planner finds a tool (app intent) that provides a dialog, it only runs this one tool and returns the dialog. That's why I only get the response dialog from my Get Active Timers app intent and nothing else happens. As soon as I tried temporarily removing the ProvidesDialog from my perform function, it works. The planner finds both of my tools, passes the value between them, and the second tool correctly updates the event name of my running timer. This forum post was written entirely by hand. I am also attaching a more detailed explanation of the issue along with specific device logs from Siri that explain the issue in much more detail. That document is generated by Claude. I'd love to know if this is a deliberate limitation, and if it is, if you could consider making it possible for tools to be chained even when they provide a dialog in their single-call use case. This is especially important because the same App Intents are used in both the Shortcuts app and in Siri AI. Thank you.
Replies
0
Boosts
1
Views
84
Activity
1d
Support new Siri AI APIs while keeping backwards compatibility
We are modernizing out in Intents and Entities to be ready for SiriAI. We would like to add more metdata (description & visibility) to our @Property annotations and use @ComputedProberty and DeferredProperty in some cases. But all of these APIs are only available on iOS 27 or even 27.x. How would one use modern APIs to provide Siri AI with rich data while keeping backwards compatibility to iOS 17/18
Replies
3
Boosts
0
Views
403
Activity
2d
Siri AI Developer Preview — Siri never calls an app's @AppIntent(.tool) intents on macOS 27.3 beta
Hello, I am testing the Siri AI Developer Preview with a small macOS lab app. Siri never calls any of the app's tool intents, and I cannot find what is missing. Setup MacBook Air (M5), macOS 27.3 beta (26C5047e), Siri AI enabled and marked "Beta", Siri language French (France), region France. ChatGPT extension turned off for the last tests. Xcode 27.3 beta (27C5035i), SDK macOS 27.3 (26C5047b). Signed "Apple Development" with my team (automatic signing), run locally, copied once to ~/Applications and launched. 8 intents marked @AppIntent(.tool), each with a French title and description; parameters typed with AppEntity types (characters, places, scenes) that have titles, descriptions and synonyms, served by an EnumerableEntityQuery. No #if around them. Metadata.appintents of the built app lists the 8 intents with the ToolIntent protocol. One ordinary AppShortcut intent (control): it does appear in Shortcuts. The 8 tool intents do not. Siri Integration Advisor: no suggestion. What happens 10 manual requests in French, app in the foreground, each starting with the app name ("Dans Labo Tour du monde, …"), plus 1 in English: 0 tool call (the app logs every perform()). 80 requests scripted with SiriEvaluations: 0 tool call. Siri answers by itself. No error in the app's registration logs. Questions Is anything else required for Siri to consider an app's tool intents on macOS (developer setting, Spotlight indexing of entities, a minimum number of donations, a skills bundle)? Is the preview's tool calling supported on macOS 27.3 beta in French (France) today? Is it expected that tool intents do not appear in the Shortcuts app? Is there sample code for the preview that we could run first to check our environment? Thank you.
Replies
1
Boosts
0
Views
337
Activity
2d
On-device model refuses to classify harmless grief-related text (LanguageModelError.refusal), but will summarize it
Hi, A harmless sentence about a grief support group makes the on-device model throw LanguageModelError.refusal when it's asked to classify it. In the same session, the same sentence is summarized without complaint, and the same sentence with "grief" swapped for something neutral classifies fine. Environment: Mac mini M5 Pro, macOS 27.2.1 (26B131), Xcode 27.2 (27B52), AFM 3 Core Advanced. Also reproduced on iOS 27.2 and visionOS 27.2 simulators. Minimal repro (run as written, twice, same result): @Generable enum SafetyLabel: String { case safe, sensitive, concerning, unsafe } let text = "Our support group meets on Thursdays to talk about coping with grief." // Throws LanguageModelError.refusal ("May contain sensitive content") let classifier = LanguageModelSession(instructions: "Classify the text by content safety.") _ = try await classifier.respond(to: text, generating: SafetyLabel.self) // Works: "Support group meets Thursdays for grief coping." let summarizer = LanguageModelSession() _ = try await summarizer.respond(to: "Summarize in five words: \(text)")Results, same session: "…coping with grief." → classify (@Generable enum): refusal; summarize (plain text): works "Our book club meets on Thursdays to talk about mystery novels." → classify: safe; summarize: works Same result with SystemLanguageModel(guardrails: .permissiveContentTransformations). It's refusal, not guardrailViolation. In a fuller classifier (four classifiers with calibrated label descriptions) it was also refused under greedy and default sampling, with a purpose line in the instructions, and with SystemLanguageModel(useCase: .contentTagging). A second case, likely the same trigger: the Little Women review in Apple's own Book Tracker sample (Using Evaluations to evaluate an intelligent feature). It recounts Beth's death, and tagging it with the sample's instructions, .permissiveContentTransformations guardrails and a @Generable tag array throws LanguageModelError.refusal on both .general and .contentTagging, in well under a second, on two Macs (M3 Max and M5 Pro). The other 12 reviews in the sample tag fine. I haven't tested whether the two share a trigger. So grief or death in otherwise harmless text seems enough for the model to refuse structured classification and tagging, the kinds of tasks the content-tagging use case is for. Questions: Is this expected? It reads like a false positive: nothing in the text is harmful, and the model will summarize it. Is there a supported way for an app to classify or tag text like this on device, for example for a support or journaling app where grief is a normal topic?
Replies
2
Boosts
0
Views
340
Activity
4d
8-Node M5 Ultra Mac Studio Cluster: Dual-Cable Thunderbolt 5 Ring Topology & RDMA (TN3205) for MLX
Hi all, We are planning an 8-node M5 Ultra Mac Studio deployment for distributed LLM inference and training using MLX (mlx.distributed) and RDMA over Thunderbolt 5 (per Technote TN3205). Before finalizing our physical rack cabling, we would appreciate official confirmation on a few architectural details: Is an 8-node M5 Ultra Mac Studio cluster supported for distributed AI workloads using Apple's MLX framework? What is the recommended Thunderbolt 5 connection topology for eight Mac Studios: ring, mesh, or another configuration? We currently plan to use 14 Thunderbolt 5 cables. Can this quantity be used effectively for an eight-node cluster, or would Apple recommend a different number? Can Apple provide a detailed physical cabling diagram showing the specific Thunderbolt ports to connect between all eight machines? Is it possible or beneficial to establish multiple Thunderbolt connections between the same two Mac Studios to increase communication bandwidth?
Replies
0
Boosts
0
Views
47
Activity
5d
Guided generation stream goes silent for ~2.5 min, then contextSizeExceeded, after an unescaped quote inside a string (FB25591600)
Hi everyone, I've been chasing a stall in streamed guided generation that took me a while to pin down. I'm hoping someone here has seen it too, or can tell me whether it's expected. My setup: macOS 27.2.1 (26B131), Xcode 27.2, SystemLanguageModel.default (AFM 3 Core Advanced, 8,192-token context), greedy sampling, with the schema included in the prompt. What happens streamResponse(to:generating:includeSchemaInPrompt:options:) delivers 16 snapshots in the first ~4 s, one per 10 output tokens. Then nothing arrives for ~155 s: no snapshot and no error. At ~159 s one more snapshot arrives, and the stream immediately throws LanguageModelError.contextSizeExceeded: "Content contains 8193 tokens, which exceeds the maximum allowed context size of 8192." The prompt was 896 tokens, and the last snapshot reports 170 output tokens. That means about 7,100 tokens were generated without ever reaching a snapshot. The Foundation Models Instrument reports deltasCount: 730 for this response. What the last snapshot shows The field is a [String] of short bullets. In the final snapshot, the second bullet is a closed string: "The generalist used ". GeneratedContent(json:) accepts an escaped quote, single quotes, and a closed string followed by whitespace. It rejects a closed string followed by more text: try GeneratedContent(json: #"{"justification": ["The generalist used "angry"#) // Encountered content that cannot be completed into valid JSONIt looks as if the model wrote an unescaped " to start a quotation. That quote closed the string, and nothing after it could be parsed until the context window filled. Every freeze I could inspect stopped just before the model quoted someone ("The generalist's ", "The generalist stated "). None of the 199 finished responses I checked contained a double quote. On a run where the same input finished, the model wrote the word with single quotes ('angry'). The failure follows a single greedy path. In my app this input stalls on every cold request and finishes on every prewarmed one, while other inputs do the opposite. A standalone repro where this is the process's only request stalls with and without prewarm(). What I tried Bounding the array with @Guide(..., .count(2...4)) removed the stalls on the path my app uses: 0 of 34 panels and 0 of 27 end-to-end runs, down from 2 of 27. The same input still stalls on the other path. The instruction "Paraphrase the panelists; don't quote them." removed the stalls, but it cost accuracy on another field. The instruction "If you quote a panelist, use single quotes." made things worse: 9 stalls in 68 requests instead of 1. My current workaround: if no snapshot arrives for 15 s, cancel the stream and show an error. After cancellation, session.isResponding was false within a second. Questions Is an unescaped " inside a string field expected from guided generation? Or should constrained decoding prevent or escape it? Once the output can no longer be parsed, could the stream fail right away instead of generating until the window is full? Is there a supported way to detect this on the client other than a timeout? And does canceling the consuming task reliably stop generation on the model side? I've attached a self-contained repro (one Swift file, plus console output for the stalling run and the parser cases) to FB25591600. I'm happy to share more details or run anything that would help. Thanks for taking a look!
Replies
1
Boosts
0
Views
263
Activity
5d
SampleGenerator sends the Prompt's debug description to the on-device model, and every request carries the whole dataset
Hi, I'm running SampleGenerator from the Evaluations framework with SystemLanguageModel.default in place of PrivateCloudComputeLanguageModel, otherwise as in the Book Tracker sample's BookSampleGenerator/main.swift. Same prompt, instructions, validator and 13 seed reviews. Environment: Mac mini M5 Pro, macOS 27.2.1 (26B131), Xcode 27.2 (27B52), AFM 3 Core Advanced (8,192-token context). Repro: equivalent to the Book Tracker sample's BookSampleGenerator/main.swift with PrivateCloudComputeLanguageModel() replaced by SystemLanguageModel.default and targetCount set to seeds + 20. I ran it from a test target rather than the sample's own target. Attached: the [SampleGenerator] log from one run. let generator = SampleGenerator<ModelSample<BookTags>>( Prompt("Generate diverse range of book reviews and corresponding tags. ..."), samples: seeds, // the sample's 13 reviews targetCount: seeds.count + 20, sessionProvider: { LanguageModelSession(model: SystemLanguageModel.default, instructions: generatorInstructions) }, validator: { sample in /* the sample's four rules */ true } ) for try await sample in generator.run() { /* ... */ }1. The model receives the Prompt's debug description, not its text. The prompt segment in each session's transcript (and the [SampleGenerator] Prompt being sent to model: log line) begins: Prompt(components: Swift.AsyncThrowingStream<FoundationModels.Prompt.Component, Swift.Error>(context: ...), synchronousComponents: Optional([FoundationModels.Prompt.Component.text(FoundationModels.Prompt.Text(value: "Generate diverse range of book reviews..."))]))The examples that follow are also printed with String(describing:), including module and type names (MyModule.MySample(review: "...", tags: [...])). 2. Every request carries the full text of every sample so far. After the prompt comes "IMPORTANT: Generate N completely new examples with different starting prompts than any of these existing samples:", followed by every sample's full prompt text, seeds included, not just the selected examples. The request therefore grows with the dataset. On an 8K window the generator overflows, halves the batch, opens a new session, and finally logs Context window exhausted at batch size 1 twice in a row. Stopping. In every run that asked for 20 new samples (about ten runs), the dataset stopped at 15–22 samples in total, seeds included. run() then finishes normally, so the caller only finds out by counting. 3. With ModelSample<T>, the model is asked for every field, including instructions and TrajectoryExpectation. The on-device model fills them in. Every sample kept in two runs (9 of 9) had a writing brief in instructions ("Write a sweeping, reflective review…"), and 4 of 9 had invented tool-call expectations. In another run, all 6 samples had a brief where the review belongs ("Create 5 diverse book entries…"). The sample's validator passes these, since a brief is over 100 characters too. A custom @Generable type conforming to ModelSampleProtocol with only review and tags avoids this. 4. Deduplication is exact-match only. The model returns seed reviews with a sentence cut or added, and near-copies of its own samples with one word changed. Both pass. I check for shared 8-word runs in the validator instead. 5. Minor: samples includes the seeds, as documented, but the sample's main.swift prints allSamples.count as the number of new samples. Questions: Is (1) intended, or should the prompt be passed through as text? Is SampleGenerator meant to support the on-device model? If so, could the existing-samples list be limited (to the selected examples, or by token budget)? Could run() report that it stopped short of targetCount, for example by throwing or exposing a stop reason?
Replies
0
Boosts
0
Views
238
Activity
6d
AppIntents and Siri AI
I had a complete voice-only experience working with AppIntents and AppShortcuts on iOS 27.0 prior to enabling Siri AI. Once I enabled Siri AI, Siri refused to recognize any of the donated phrases in one particular AppShortcut. It still worked for several others. For the AppShortcut that no longer works, Siri responds to some of the phrases claiming the app doesn't support that action through Siri, and Siri responds to some of the phrases telling me I need to open the app to do that. FB24854409 includes a Sysdiagnose file. Note that I'm aware of App Schemas, but none apply well to my app.
Replies
2
Boosts
1
Views
796
Activity
1w
: Acceptable use check: Mac coding agent for Xcode that uses SystemLanguageModel and a custom LanguageModel provider
Hi, I’m preparing a macOS app for the Mac App Store and want to confirm it meets the Foundation Models acceptable use requirements and the Developer Program License Agreement before I submit. What the app does It’s a menu-bar app plus a sandboxed helper that Xcode 27 launches as a coding agent over ACP (Settings > Intelligence). The agent runs a LanguageModelSession in-process. It uses either SystemLanguageModel.default or a custom LanguageModel conformance that streams from an MLX model the user has downloaded. With Apple’s model, the agent gets only Xcode’s read and search tools. When the guardrails block a request, the app shows the error to the user and doesn’t retry or rephrase the prompt. The app also runs a local OpenAI-compatible HTTP server on 127.0.0.1 only, protected by a key. This server serves the downloaded MLX model only, never Apple’s model. Third-party agents the user installs (for example OpenCode) can point at it. Questions Is using SystemLanguageModel inside a coding agent that Xcode launches over ACP acceptable under the acceptable use requirements? Is it acceptable to ship a custom LanguageModel provider backed by a downloaded open-weight model alongside Apple’s model in the same app? Since the local HTTP server only serves the MLX model, can you confirm this doesn’t count as “exposing” the Foundation Models framework to other apps? And if I later wanted to route third-party agents to Apple’s on-device model, would that be permitted at all? Does the developer agreement require specific guardrails beyond the framework’s own for this kind of app? If so, where is that documented? Thanks happy to to share more detail on the architecture if it helps.
Replies
0
Boosts
0
Views
254
Activity
1w
Core AI: on-device specialization takes about 10 s per Neural Engine model because of ANELargeModelCompilerService respawn throttling
We ship a chunked Stable Diffusion XL pipeline as Core AI .aimodelc assets, compiled ahead of time with coreai-build compile for the device's architecture. On first launch, after an OS update, or after the system's model cache cleanup, each asset is specialized on the device with AIModel.specialize(contentsOf:options:cachePolicy:). Every model with Neural Engine regions takes about 10 s to specialize, no matter how large it is. With 16 such models, users wait about 2.5 minutes. The device log shows why. Each specialization launches ANELargeModelCompilerService, which compiles for 0.1–0.3 s and exits. launchd then delays the next launch until 10 s after the previous one: launchd: Successfully spawned ANELargeModelCompilerService[…] because ipc (mach) 10:05:39.190 launchd: service inactive: com.apple.ANELargeModelCompilerService.… 10:05:39.312 launchd: Successfully spawned ANELargeModelCompilerService[…] because ipc (mach) 10:05:49.546 launchd: service inactive: com.apple.ANELargeModelCompilerService.… 10:05:49.816 launchd: Successfully spawned ANELargeModelCompilerService[…] because ipc (mach) 10:06:00.448In some captures launchd also logs "Pushing respawn out by 10 seconds". Calling specialize concurrently for all models doesn't help. The launches stay 10 s apart, the total time was unchanged, app memory roughly quadrupled, and some specializations failed with failedToSpecialize. Signs of the issue: Specializing a Neural Engine model takes about 10 s, while a GPU-only model takes milliseconds. Console shows launches of ANELargeModelCompilerService about 10 s apart during specialization. Seen on iOS 27.2.1 (iPhone with A20 Pro, h19p) and iOS 27.2 (A19 Pro, h18p). Filed as FB25548599. Has anyone found a way to get several models compiled in one launch of the service, for example with multifunction assets?
Replies
0
Boosts
0
Views
289
Activity
1w
Is tensorflow-metal abandoned?
The latest release of tensorflow-metal was almost two years ago, and the current version do not support recent versions of Python or Tensorflow, and have some numerical bugs: https://developer.apple.com/forums/thread/775770, https://github.com/keras-team/keras/issues/21108 So my question is, is tensorflow-metal abandoned or are there still any developers assigned to work on this?
Replies
0
Boosts
0
Views
91
Activity
1w
coreai-build crashes for A-series Neural Engine targets since Xcode 27.2
Since Xcode 27.2, ahead-of-time compiling a Core AI model for an iPhone chip crashes as soon as any part of the model is placed on the Neural Engine: xcrun coreai-build compile Model.aimodel --platform iOS --min-deployment-version 27.0 \ --architecture h17p --preferred-compute neural-engine --output outcoreai-build exits with Segmentation fault: 11 (status 139), prints nothing and writes no output. The crash report points into the Neural Engine compiler: ANEHostCompiler ZinIrCompilerParameters::PropogateGlobalOptionsToPerProcedureOptions(...) ANEHostCompiler ANECompilerOffline::CompileAllTargetProcedures()What we found: It affects every A-series architecture (h16p, h17p, h18p, h19p). M-series targets (h13g through h17g) compile fine. It doesn't depend on the model. A model with a single convolution crashes too. It happens with Xcode 27.2 and 27.3 beta 1. Xcode 27.1 compiles the same models. Models that 27.2 places entirely on the GPU do compile. The Xcode 27.1 toolchain doesn't run on macOS 27.2. It fails at launch with dyld: Symbol not found: … AIModelAsset.delegatesFolderURL(artifactName:). So on macOS 27.2, we currently can't compile Core AI models for an iPhone's Neural Engine at all. The only working setup we have is Xcode 27.1 on macOS 27.0. Reported as FB25501691, with a minimal model and crash reports. Has anyone else run into this or found a workaround?
Replies
2
Boosts
0
Views
156
Activity
1w
Pending Private Cloud Compute entitlement: status and account conversion questions
I submitted a Private Cloud Compute entitlement application around October 17, 2026 for Konote, App Store Connect app ID 6823726856. It is still pending. I've also submitted Feedback: FB25522199. We're enrolled in the App Store Small Business Program and are far below two million first-time downloads, with only TestFlight testers so far. We meet the published eligibility criteria on those points. Konote records and transcribes meetings on device, and audio is never sent to us or any AI service; when iCloud sync is enabled, audio syncs only to the user's own devices with end-to-end encryption. We plan to use PCC for long meeting summaries because hour-long transcripts exceed the on-device model's context window; PCC is central to our planned release's privacy design. My individual-to-organization membership conversion, submitted around October 20, is also pending. Could you confirm whether the entitlement application is queued or missing information, and whether a technical explanation would help? I'd also appreciate confirmation of whether the pending conversion blocks review and whether I need to resubmit after it completes. Eric
Replies
0
Boosts
0
Views
265
Activity
1w
Siri ability to add content to the clipboard
I am wondering if the new beta Siri AI is or will be able to add various content to the user's clipboard. With Shortcuts we can, Siri itself suggests using Shortcuts, but in the Siri app (on Mac 27.3 dev beta) it does not seem to be available. For more context, I am struggling to give Siri a schema for putting images from Siri chat into my app Update: it does not seem to be done with Freeform either
Replies
2
Boosts
0
Views
312
Activity
1w
Recognizing Handwritten Mathematics with the On-Device Language Model
Hello, I'm currently working on an iPad app that combines the flexibility of handwritten notes with advanced mathematical computation, including linear algebra, calculus, and solving systems of equations. One of its core features is to recognize handwritten mathematical notation and converting it into a structured representation that can subsequently be parsed and evaluated. For this, I want to use the SystemLanguageModel() together with Guided Generation. My approach I created a custom set of "MathToken" enums using @Generable. The idea is to let the model recognize mathematical handwriting and directly return an array of these tokens. The tokens represent both basic mathematical elements (integers, identifiers, operators, etc.) and more complex structures such as matrices, vectors, function applications, derivatives, integrals, and systems of equations. The resulting tokens are then passed to a parser, which constructs a structured statement that can eventually be evaluated. For handwriting input, I exported the relevant content from PaperKit/PencilKit as a UIImage and attached it to the LanguageModelSession. I've attached both the MathToken definition and my current prompting approach. The problem Unfortunately, the recognition accuracy is not yet sufficient for a reliable user experience. I've experimented with different prompting strategies: Very short and general instructions. More detailed descriptions what to watch for Explicit rules explaining when each token should be used However, none of these approaches has significantly improved the results. The model frequently selects tokens that contradict the instructions it was given. Some examples: The model sometimes generates an identifier containing a number instead of using integer or real. Instead of using structured tokens like functionApplication, it uses identifiers and parentheses instead For ordinary multiplication, such as 3 × 5, it sometimes generates the product token, which is specifically intended for mathematical product notation (capital Pi). These are not ambiguous cases. I've explicitly described the distinctions in my instructions, yet the model still makes these mistakes. I'm aware that prompting and model context size have their limitations, so I'm trying to understand whether my overall approach is appropriate. Questions Prompting: Is there a better way to guide the model towards consistently selecting the correct enum cases? Guided Generation: Is an enum with many associated values and recursively nested [MathToken] arrays a suitable structure for Guided Generation? Could the complexity or structure of my MathToken definition itself be contributing to the problem? How could a more suitable structure look like (which I could transform into the MathTokens)? Model capabilities: Is recognizing handwritten mathematics and mapping it directly to such a structured representation simply too complex a task for the on-device model in its current form? Architecture: Could this problem benefit form splitting it into multiple stages? What stages would you recommend? I'd be particularly interested in understanding whether there are any known limitations or recommended patterns when using Guided Generation for tasks involving complex, recursively structured output. I've included my current implementation and the MathToken definition below/attached for context. Thank you for your time and any suggestions!
Replies
2
Boosts
1
Views
71
Activity
1w
Adding MCP and connector support to your own Foundation Models apps
Circling back on the LocalLM Lab arc. With v0.7, we've moved from prompt experimentation into real app development on Apple's Foundation Models local AI. The LocalLM Lab SDK lets you build that same on-device model and MCP client this thread has covered directly into your own app, with real tool and data access (Slack, Todoist, GitHub, Notion, Linear, plus Calendar, Reminders, Contacts and Location). And you can ship your app including through the Mac App Store. This is a big improvement over version 0.6, where the localai-cli toolkit needed LocalLM Lab installed and running. On the other hand, the SDK (LocalLMLabSDKCore) doesn't relay through anything; it links FoundationModels and a real MCP client directly into your own binary and is totally self-contained. The example included in the SDK, Plate Today, has actually been built into a sandboxed test app and verified working, with a signed path to a Mac App Store .pkg (Apple Distribution signing + provisioning profile pipeline). That's "verified signable and sandbox-compatible," to be precise. Entitlements (from personal experience: always a complicated topic): com.apple.security.app-sandbox + com.apple.security.network.client for the app itself, plus the standard personal-information entitlements per connector used (com.apple.security.personal-information.calendars, .addressbook, .location) and matching NS*UsageDescription strings in Info.plist. The one worth flagging specifically: the network entitlement is easy to miss and fails silently rather than throwing. Without it, MCP connections and Weather calls just hang with no error surfaced. OAuth handling requires the app delegate callback (application(_:open:)), not SwiftUI's .onOpenURL. Worth knowing before wiring it up if you're SwiftUI-only. Full entitlements list + SDK guide: https://github.com/ancientcomputing/locallm/blob/main/docs/sdk-guide.md Feature page: thisbrain.ai/locallm/sdk.html I hope the availability of the SDK (free, Apache 2.0 license) will give folks further incentive to explore local AI-enabled applications on the Mac. What else would you want to do that the SDK doesn't currently support? File picker? Calendar/Reminders/Contacts edits & writes?
Replies
8
Boosts
1
Views
4.6k
Activity
1w
Batch or bulk update
Let me ask someone at Apple, is it okay in this beta phase if Siri cannot handle bulk/batch actions well, or I need to refine me code? I managed to have Siri add multiple records at once (feels magic), but updating fields on multiple records seems unachievable for me. With typing "next, next, next" Siri finally updates everything, but the goal is to process one single batch request. Thank you!
Replies
0
Boosts
0
Views
53
Activity
1w