In this video tutorial, we'll go through a Colab notebook that shows you how to crawl a website and turn it into a GPT-4 enabled AI assistant.
In this case, I've crawled my own website but you could apply it to many other use cases that involve augmenting GPT-4 with a separate body of knowledge or external database.
In the notebook, the steps I'll walk through include:
- Scraping my own site MLQ.ai
- Convert the text from each article into embeddings using the OpenAI API
- Store these embeddings in a vector database: Pinecone
- Use GPT-4 to query the site, answer with context, and return relevant article sources