Deep Research Technology Overview

Introduction

• Chat GPT initially had a static knowledge model with a cut-off date for information.
• Evolution towards large language models (LLMs) that access real-time internet data.
• Introduction of "Deep Research" as a new technology for autonomous data retrieval and report generation.

What is Deep Research?

• A service offered by OpenAI and Google, among others, enabling independent online research by agents.
• It executes tasks like accessing the internet to independently gather and process data.
• Capable of a rinse and repeat process for iterative data collection and analysis.

Available Services

• OpenAI & Google: Paid subscriptions to access their Deep Research capabilities.
  • OpenAI: $200/month for Pro subscribers.
  • Google: ~$20-$25/month for Gemini Advanced subscribers.
• Perplexity: Offers a free tier with a limited number of queries per day.
• David Zang’s Open-Source Initiative: A free, self-hosted option combining search engine results and web scraping.

Open-Source Deep Research by David Zang

• Open-source project accessible via a GitHub repository.
• Works on Linux and likely MacOS; untested on Windows.
• Requires Node.js and Docker for setup.
• Needs two API keys:
  1. Fir Craw API Key for search and content extraction.
  2. OpenAI API Key for LLM functionality.
• Self-hosting: Options to self-host Fir Craw for cost-saving; can be complex to manage Google search limits.

Running Large Language Models Locally

• Potential to run LLMs locally using tools like Olama.
• Challenges with port configuration and speed of data processing for extensive searches.

Installation

• Follow simple steps involving cloning a GitHub repository and running Docker commands.
• Configure API keys for required services.
• Tested primarily on Linux systems.

Demonstrations

• Demonstration involves setting up Docker and running Deep Research queries.
• Examples included finding the best F1 driver with iterations refining results.

Perplexity Deep Research

• A web-based platform offering an integrated Deep Research system.
• Enables users to conduct research without direct intervention by leveraging AI.

Conclusion

• Deep Research leverages AI capabilities to autonomously collect and analyze data.
• Available in various formats from free open-source to paid subscriptions offering different features.
• Can be applied to diverse fields such as economics, philosophy, technology, and more.

Final Thoughts

• Deep Research is a promising tool for comprehensive, independent research.
• Users should evaluate services based on budget, needs, and desired depth of analysis.
• The community's feedback is valuable in assessing effectiveness across different platforms.

---

Ensure to check the individual service's terms and conditions for the most updated pricing and availability information.