-
Notifications
You must be signed in to change notification settings - Fork 1.5k
feat(apple-ai): support Apple's Foundation Models framework with local-first and cloud Gemini fallback #1855
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
peterfriese
merged 10 commits into
wwdc26-preview
from
peterfriese/firebase-ai/quickstart
Jul 2, 2026
Merged
Changes from all commits
Commits
Show all changes
10 commits
Select commit
Hold shift + click to select a range
04efb3a
feat(apple-ai): add support for Apple Foundation Models integration
peterfriese 9ad90c8
refactor(apple-ai): clean up tasks and session handling in AppleAIVie…
peterfriese 7cd57d2
feat(apple-ai): add support for Google Maps grounding attributions in…
peterfriese cfb49d8
feat(apple-foundation-models): add model indicator to all tabs
peterfriese 784a95a
feat(apple-foundation-models): implement local model fallback for pla…
peterfriese 0de49f0
fix(apple-foundation-models): ensure image attachment is sent in clou…
peterfriese 48bafcb
Refactor Foundation Models: split ViewModels, inject dynamic backend,…
peterfriese 424d6d8
refactor(apple-ai): clean up unused variables, simplify error handlin…
peterfriese 11bfbe5
fix(apple-ai): fix selected backend propagation and simplify async ta…
peterfriese 8e6da5b
refactor(apple-ai): change ConversationKit to remote and remove redun…
peterfriese File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
38 changes: 38 additions & 0 deletions
38
firebaseai/FirebaseAIExample/Features/AppleFoundationModels/Models/Models.swift
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,38 @@ | ||
| // Copyright 2026 Google LLC | ||
| // | ||
| // Licensed under the Apache License, Version 2.0 (the "License"); | ||
| // you may not use this file except in compliance with the License. | ||
| // You may obtain a copy of the License at | ||
| // | ||
| // http://www.apache.org/licenses/LICENSE-2.0 | ||
| // | ||
| // Unless required by applicable law or agreed to in writing, software | ||
| // distributed under the License is distributed on an "AS IS" BASIS, | ||
| // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| // See the License for the specific language governing permissions and | ||
| // limitations under the License. | ||
|
|
||
| import Foundation | ||
| import FoundationModels | ||
|
|
||
| @available(iOS 26.0, *) | ||
|
peterfriese marked this conversation as resolved.
|
||
| @Generable | ||
| struct IdentifiedObject: Equatable, Codable { | ||
| @Guide(description: "The name of the primary object or landmark detected.") | ||
| let name: String | ||
|
|
||
| @Guide( | ||
| description: "The category of the object (e.g. Landmark, Plant, Food, Animal, Device, Clothing)." | ||
| ) | ||
| let category: String | ||
|
|
||
| @Guide(description: "A short, 2-sentence description of the object and interesting facts.") | ||
| let description: String | ||
| } | ||
|
|
||
| @available(iOS 26.0, *) | ||
| @Generable | ||
| struct TextSummary: Equatable, Codable { | ||
| @Guide(description: "A list of exactly 2 key summary points.") | ||
| let summaryPoints: [String] | ||
| } | ||
30 changes: 30 additions & 0 deletions
30
firebaseai/FirebaseAIExample/Features/AppleFoundationModels/Screens/HybridAIScreen.swift
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,30 @@ | ||
| // Copyright 2026 Google LLC | ||
| // | ||
| // Licensed under the Apache License, Version 2.0 (the "License"); | ||
| // you may not use this file except in compliance with the License. | ||
| // You may obtain a copy of the License at | ||
| // | ||
| // http://www.apache.org/licenses/LICENSE-2.0 | ||
| // | ||
| // Unless required by applicable law or agreed to in writing, software | ||
| // distributed under the License is distributed on an "AS IS" BASIS, | ||
| // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| // See the License for the specific language governing permissions and | ||
| // limitations under the License. | ||
|
|
||
| import SwiftUI | ||
|
|
||
| @available(iOS 27.0, *) | ||
| struct HybridAIScreen: View { | ||
| @StateObject private var viewModel: HybridAIViewModel | ||
|
|
||
| init(backendType: BackendOption) { | ||
| _viewModel = StateObject(wrappedValue: HybridAIViewModel(backendType: backendType)) | ||
| } | ||
|
|
||
| var body: some View { | ||
| FoundationModelsContainer(viewModel: viewModel, title: "Hybrid AI") { vm in | ||
| HybridAIView(viewModel: vm) | ||
| } | ||
| } | ||
| } |
32 changes: 32 additions & 0 deletions
32
firebaseai/FirebaseAIExample/Features/AppleFoundationModels/Screens/VisionIDScreen.swift
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,32 @@ | ||
| // Copyright 2026 Google LLC | ||
| // | ||
| // Licensed under the Apache License, Version 2.0 (the "License"); | ||
| // you may not use this file except in compliance with the License. | ||
| // You may obtain a copy of the License at | ||
| // | ||
| // http://www.apache.org/licenses/LICENSE-2.0 | ||
| // | ||
| // Unless required by applicable law or agreed to in writing, software | ||
| // distributed under the License is distributed on an "AS IS" BASIS, | ||
| // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| // See the License for the specific language governing permissions and | ||
| // limitations under the License. | ||
|
|
||
| import SwiftUI | ||
| import PhotosUI | ||
|
|
||
| @available(iOS 27.0, *) | ||
| struct VisionIDScreen: View { | ||
| @StateObject private var viewModel: VisionIDViewModel | ||
| @State private var photosPickerItem: PhotosPickerItem? = nil | ||
|
|
||
| init(backendType: BackendOption) { | ||
| _viewModel = StateObject(wrappedValue: VisionIDViewModel(backendType: backendType)) | ||
| } | ||
|
|
||
| var body: some View { | ||
| FoundationModelsContainer(viewModel: viewModel, title: "Vision ID") { vm in | ||
| VisionIDView(viewModel: vm, photosPickerItem: $photosPickerItem) | ||
| } | ||
| } | ||
| } |
58 changes: 58 additions & 0 deletions
58
...seAIExample/Features/AppleFoundationModels/ViewModels/FoundationModelsBaseViewModel.swift
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,58 @@ | ||
| // Copyright 2026 Google LLC | ||
| // | ||
| // Licensed under the Apache License, Version 2.0 (the "License"); | ||
| // you may not use this file except in compliance with the License. | ||
| // You may obtain a copy of the License at | ||
| // | ||
| // http://www.apache.org/licenses/LICENSE-2.0 | ||
| // | ||
| // Unless required by applicable law or agreed to in writing, software | ||
| // distributed under the License is distributed on an "AS IS" BASIS, | ||
| // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| // See the License for the specific language governing permissions and | ||
| // limitations under the License. | ||
|
|
||
| import Foundation | ||
| import SwiftUI | ||
| import Combine | ||
| import FoundationModels | ||
| import FirebaseAILogic | ||
|
|
||
| enum ModelPreference: String, CaseIterable, Identifiable { | ||
| case auto = "Auto (Local First)" | ||
| case cloud = "Cloud Only" | ||
|
|
||
| var id: String { rawValue } | ||
| } | ||
|
|
||
| @available(iOS 27.0, *) | ||
| @MainActor | ||
| class FoundationModelsBaseViewModel: ObservableObject { | ||
| @Published var inProgress = false | ||
| @Published var error: Error? | ||
| @Published var modelPreference: ModelPreference = .auto | ||
| @Published var isUsingLocalModel: Bool = false | ||
|
|
||
| internal var activeTask: Task<Void, Never>? | ||
| let backendType: BackendOption | ||
|
|
||
| init(backendType: BackendOption) { | ||
| self.backendType = backendType | ||
| } | ||
|
|
||
| func stopActiveTask() { | ||
| activeTask?.cancel() | ||
| activeTask = nil | ||
| inProgress = false | ||
| } | ||
|
|
||
| internal func getFirebaseAI() -> FirebaseAI { | ||
| switch backendType { | ||
| case .googleAI: | ||
| return FirebaseAI.firebaseAI(backend: .googleAI()) | ||
| case .vertexAI: | ||
| // Using "global" as location for Foundation Models fallback compatibility | ||
| return FirebaseAI.firebaseAI(backend: .vertexAI(location: "global")) | ||
| } | ||
| } | ||
| } |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.