Exploring Google AI Edge Gallery: Running Large Language Models Locally on Android

Introduction

• Interest in running large language models locally on devices instead of the cloud.
• Google has launched the Google AI Edge Gallery, an Android app allowing generative AI models to run on smartphones without internet.

Key Features of Google AI Edge Gallery

• Local Execution: AI models run entirely on the Android device, no internet required post-download.
• Model Exploration: Users can experiment with different generative models.
• Image and Vision Capabilities: Incorporates prompts and image functionalities.
• AI Chat: Includes standard chat interaction with LLMs.

Gemma 3 and Gemma 3N Models

• Gemma 3N: Optimized for everyday devices (e.g., smartphones, laptops).
• Parameter Efficiency: Uses reduced set of effective parameters, divided into text, visual, audio, and per-layer embedding groups.
• Models like E2B and E4B operate efficiently with fewer parameters, saving resources.

Technical Details

• PLE (Per Layer Embedding): Enhances model layer performance by generating data outside of memory.
• Matt Former Architecture: Allows nested smaller models within a larger model to run only necessary components, reducing compute cost.

Installation Process

• Not available on Google Play Store.
• Download APK from Google’s GitHub repository.
• Installation requires using ADB (Android Debug Bridge) for devices with Google Play Protect enabled.

Demonstration

• App Structure: Three main sections - image inquiries, AI chat, and prompt lab.
• Model Usage: Users can select from multiple Gemma models with varying parameters to interact with.
• Performance: Various tests show models running efficiently on device hardware, with features like CPU and GPU usage stats displayed.

Conclusion and User Engagement

• Encouragement for users to experiment with the app and share experiences.
• Contact and interaction invitation through comments and subscriptions.

Additional Resources

• Mention of further learning materials and support channels.

Author: Gary Sims
Channel: Gary Explains
Additional Support: Patreon page invitation.
[Music]