Top Related Projects
Stable Diffusion with Core ML on Apple Silicon
Swift for TensorFlow
Core ML tools contain supporting tools for Core ML model conversion, editing, and validation.
Quick Overview
The google-gemini/generative-ai-swift repository is an official Swift SDK for Google's Generative AI models, including Gemini. It provides a convenient way for iOS and macOS developers to integrate Google's powerful AI capabilities into their Swift applications, enabling features like text generation, image analysis, and multimodal interactions.
Pros
- Official SDK from Google, ensuring reliability and up-to-date support
- Seamless integration with Swift and Apple platforms (iOS, macOS)
- Supports various Gemini models, including text, vision, and multimodal capabilities
- Well-documented with clear examples and usage guidelines
Cons
- Limited to Google's Generative AI models, not a general-purpose AI library
- Requires API key and potential usage costs for production applications
- May have limitations based on Google's API policies and quotas
- Relatively new, so the ecosystem and community support are still growing
Code Examples
- Initializing the GenerativeAI client:
import GoogleGenerativeAI
let apiKey = "YOUR_API_KEY"
let config = GenerationConfig(temperature: 0.9, topP: 1, topK: 1, maxOutputTokens: 2048)
let model = GenerativeModel(name: "gemini-pro", apiKey: apiKey, generationConfig: config)
- Generating text with the Gemini model:
let prompt = "Write a short story about a robot learning to paint."
let response = try await model.generateContent(prompt)
if let text = response.text {
print(text)
}
- Analyzing an image with Gemini Vision:
let image = UIImage(named: "example.jpg")!
let prompt = "Describe what you see in this image."
let response = try await model.generateContent(prompt, image: image)
if let description = response.text {
print(description)
}
Getting Started
- Install the SDK using Swift Package Manager. Add the following to your
Package.swift
file:
dependencies: [
.package(url: "https://github.com/google/generative-ai-swift", from: "0.1.0")
]
- Import the library in your Swift file:
import GoogleGenerativeAI
- Initialize the client with your API key and start using the Generative AI features:
let apiKey = "YOUR_API_KEY"
let model = GenerativeModel(name: "gemini-pro", apiKey: apiKey)
let response = try await model.generateContent("Hello, Gemini!")
print(response.text ?? "No response")
Competitor Comparisons
Stable Diffusion with Core ML on Apple Silicon
Pros of ml-stable-diffusion
- Focuses on stable diffusion models, offering specialized image generation capabilities
- Optimized for Apple devices, leveraging Core ML for efficient performance
- Provides a comprehensive implementation of the stable diffusion pipeline
Cons of ml-stable-diffusion
- Limited to image generation tasks, unlike the broader AI capabilities of generative-ai-swift
- Requires more domain-specific knowledge to use effectively
- May have a steeper learning curve for developers new to stable diffusion concepts
Code Comparison
ml-stable-diffusion:
let pipeline = try StableDiffusionPipeline(resourcesAt: resourcesURL, configuration: configuration, disableSafety: false)
let images = try pipeline.generateImages(prompt: "A cute cat", imageCount: 1)
generative-ai-swift:
let model = GenerativeModel(name: "gemini-pro", apiKey: "YOUR_API_KEY")
let response = try await model.generateContent("Describe a cute cat")
print(response.text)
The ml-stable-diffusion code focuses on image generation, while generative-ai-swift provides a more general-purpose AI interaction. The former requires more setup and configuration, while the latter offers a simpler interface for diverse AI tasks.
Swift for TensorFlow
Pros of Swift for TensorFlow
- More comprehensive machine learning framework with broader capabilities
- Deeper integration with TensorFlow ecosystem and tools
- Larger community and more extensive documentation
Cons of Swift for TensorFlow
- Steeper learning curve for developers new to TensorFlow
- Potentially more complex setup and configuration
- Less focused on generative AI specifically
Code Comparison
Swift for TensorFlow:
import TensorFlow
let model = Sequential {
Dense(units: 64, activation: relu)
Dense(units: 10, activation: softmax)
}
Generative AI Swift:
import GenerativeAI
let model = GenerativeModel(name: "gemini-pro")
let response = try await model.generateContent(prompt)
Swift for TensorFlow provides a more low-level approach to building neural networks, while Generative AI Swift offers a higher-level API specifically for generative AI tasks. The former gives more control over model architecture, while the latter simplifies interaction with pre-trained models like Gemini.
Generative AI Swift is more focused on ease of use for generative AI applications, making it quicker to implement for specific use cases. However, Swift for TensorFlow offers more flexibility and power for a wider range of machine learning tasks beyond just generative AI.
Core ML tools contain supporting tools for Core ML model conversion, editing, and validation.
Pros of coremltools
- Broader scope: Supports a wide range of machine learning models and frameworks
- Mature ecosystem: Well-established with extensive documentation and community support
- Cross-platform: Enables deployment on various Apple platforms (iOS, macOS, tvOS, watchOS)
Cons of coremltools
- Limited to Apple ecosystem: Not suitable for non-Apple platforms
- Steeper learning curve: Requires understanding of Core ML concepts and Apple's development environment
Code Comparison
coremltools:
import coremltools as ct
model = ct.convert('model.h5', source='keras')
model.save('MyModel.mlmodel')
generative-ai-swift:
import GenerativeAI
let model = GenerativeModel(name: "gemini-pro")
let response = try await model.generateContent("Hello, world!")
Summary
coremltools offers a comprehensive solution for deploying machine learning models across Apple platforms, with broad framework support and a mature ecosystem. However, it's limited to the Apple ecosystem and has a steeper learning curve. generative-ai-swift, on the other hand, provides a more focused and straightforward approach for integrating Google's Gemini models into Swift applications, but with a narrower scope and less flexibility in terms of supported models and platforms.
Convert
designs to code with AI
Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.
Try Visual CopilotREADME
Google AI SDK for Swift
The Google AI Swift SDK is the easiest way for Swift developers to build with the Gemini API. The Gemini API gives you access to Gemini models created by Google DeepMind. Gemini models are built from the ground up to be multimodal, so you can reason seamlessly across text, images, and code.
[!CAUTION] The Google AI SDK for Swift is recommended for prototyping only. If you plan to enable billing, we strongly recommend that you use a backend SDK to access the Google AI Gemini API. You risk potentially exposing your API key to malicious actors if you embed your API key directly in your Swift app or fetch it remotely at runtime.
Get started with the Gemini API
- Go to Google AI Studio.
- Login with your Google account.
- Create an API key. Note that in Europe the free tier is not available.
- Check out this repository.
git clone https://github.com/google/generative-ai-swift
- Open and build the sample app in the
Examples
folder of this repo. - Run the app once to ensure the build script generates an empty
GenerativeAI-Info.plist
file - Paste your API key into the
API_KEY
property in theGenerativeAI-Info.plist
file. - Run the app
- For detailed instructions, try the Swift SDK tutorial on ai.google.dev.
Usage example
-
Add
generative-ai-swift
to your Xcode project using Swift Package Manager. -
Import the
GoogleGenerativeAI
module
import GoogleGenerativeAI
- Initialize the model
let model = GenerativeModel(name: "gemini-1.5-flash-latest", apiKey: "YOUR_API_KEY")
- Run a prompt
let cookieImage = UIImage(...)
let prompt = "Do these look store-bought or homemade?"
let response = try await model.generateContent(prompt, cookieImage)
For detailed instructions, you can find a quickstart for the Google AI SDK for Swift in the Google documentation.
This quickstart describes how to add your API key and the Swift package to your app, initialize the model, and then call the API to access the model. It also describes some additional use cases and features, like streaming, counting tokens, and controlling responses.
Logging
To enable additional logging in the Xcode console, including a cURL command and
raw stream response for each model request, add
-GoogleGenerativeAIDebugLogEnabled
as Arguments Passed On Launch
in the
Xcode scheme.
Command Line Tool
A command line tool is available to experiment with Gemini model requests via Xcode or the command line:
open Examples/GenerativeAICLI/Package.swift
- Run in Xcode and examine the console to see the options.
- Edit the scheme's
Arguments Passed On Launch
with the desired options.
Documentation
See the Gemini API Cookbook or ai.google.dev for complete documentation.
Contributing
See Contributing for more information on contributing to the Google AI SDK for Swift.
Developers who use the PaLM SDK for Swift (Decommissioned)
[!IMPORTANT] The PaLM API is now decommissioned. This means that users cannot use a PaLM model in a prompt, tune a new PaLM model, or run inference on PaLM-tuned models.
Note: This is different from the Vertex AI PaLM API, which is scheduled to be decommissioned in October 2024.
ââIf you're using the PaLM SDK for Swift, migrate your code to the Gemini API
and update your app's generative-ai-swift
dependency to version 0.4.0
or
higher. For more information on migrating from PaLM to Gemini, see the
migration guide.
License
The contents of this repository are licensed under the Apache License, version 2.0.
Top Related Projects
Stable Diffusion with Core ML on Apple Silicon
Swift for TensorFlow
Core ML tools contain supporting tools for Core ML model conversion, editing, and validation.
Convert
designs to code with AI
Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.
Try Visual Copilot