Process of Training AI Chatbot on Your Data Using ChatGPT API, A Guide to Building a Custom AI Chatbot

Chatbots are very important for your business or venture, We’re talking about a really smart ChatGPT chatbot that understands everything about your business and can help customers by answering their questions tirelessly 24/7.

What is ChatGPT and Ai chatbots.

In simple English, ChatGPT is a smart computer program that talks like a person. AI chatbots, like ChatGPT, help us communicate with computers in a more natural way.

But don’t worry here is our comprehensive article on what’s ChatGPT and ChatGPT-4, to help youunderstand ChatGPT and it’s basics.

Nevertheless, In an earlier article, we showed how to make a personalized AI chatbot using the ChatGPT API and assign a role to personalise it.

But what if you want to train it with your own information, like a book or financial data? Our new article explains how to do this easily with Lang Chain and ChatGPT API.

Things to NOTE before you Train your AI chatbot with Your Data.

You can train the AI chatbot on any platform, like Windows, macOS, Linux, or ChromeOS. I’m using Windows 11 here, but the steps are similar for others.

For training your own AI Chatbot, a powerful computer is recommended, but a basic one works for testing. I trained a model on a Chromebook with a 100-page book (~100MB). For larger data sets, opt for a strong computer.

The guide is simple for anyone to follow, even with basic computer knowledge. You can create a Q&A AI chatbot in minutes, no coding required. If you read our previous article, it’ll be even easier.

The data should be in English for best results, but OpenAI confirms it works well with languages like French, Spanish, German, etc. Give it a try in your own language!.

After building your personalised AI chatbots, you will be able to integrate it to your business website, Apps, Servers and Systems.

Here is an example of AI chatbot in action on a website.

Impotence of Custom-trained ChatGPT AI chatbot for your business and brand.

It improves your customer experiences.

Just imagine your website visitors exploring your site when they are warmly welcomed by a helpful AI chatbot, eager to help them in understand your business. They receive all the necessary information through an enjoyable, interactive conversation. This is what ChatGPT AI chatbot can do.

Helps in capturing Leads like a pro.

Let’s say for example you have a cars dealer company. Someone visits your website looking for the top 10 largest cars to buy a house in 2023. Instead of leaving them to search on their own, your AI chatbot helps by giving them information about the right areas based on what they want and how much they can spend and even book a call session for them with you.

Useful in keeping your customers engaged.

Imagine a visitor to your website while reading a blog recipe about making eggless cake, your ChatGPT AI chatbot pops up with a message, “Did you know you can make an eggless cake using just a banana? Click here to learn more!”. Clicking the link leads to another post, making them stay on your site longer.

To Help with customer support.

Instead of answering to customer questions alone, your custom AI chatbot will answer their questions instantly since it understands your business better.

To improve the internal employee experience

Instead of spending hours sifting through company documents or waiting for email responses from the HR team, employees can easily engage with this chatbot to obtain the information they require.

These are all incredible right!?

Now let’s prepare for an exciting dive into the world of AI as we uncover the tools and techniques to create your own revolutionary ChatGPT chatbot.

Get ready fam!

The future of AI chatbots has arrived and it starts with your data.

Step 1. Establish the software environment in order to train an AI chatbot.

Install Python and Pip.

You can download Python from the official Python website. And upgrade the pip that cones with it.

You can also install Python and Pip on your computer using the instructions provided in the linked guide.Run the setup file and make sure to check the box for “Add Python.exe to PATH” while installing.

verify that Python is correctly installed, access the Terminal on your computer. After opening it, run the below commands individually; this will display their respective version numbers. For Linux and macOS, you must now use python3 instead of python.

python --version

pip --version

To update Pip to the latest version, run the command provided below.

python -m pip install -U pip

Step 2: Install OpenAI, GPT Index, PyPDF2, and Gradio Libraries.

Now open the Terminal and run the below command to install the OpenAI library.

pip install openai

Next, let’s install GPT Index.

pip install gpt_index==0.4.24

Following that, ensure the installation of PyPDF2 and PyCryptodome for parsing PDF files.

pip install PyPDF2

pip install PyCryptodome

Lastly is the installation of Gradio library, as it facilitates the creation of a user-friendly interface for interacting with the trained AI chatbot.

pip install gradio

Step 3. Download a Code Editor.

To edit the code, use Notepad++ on Windows. Simply download and install it via the link ( Download ). If you prefer, you can also use VS Code on any platform if you are comfortable with powerful IDEs. For macOS and Linux, Sublime Text ( Download )is another good option.

In the case of ChromeOS, the effective Caret app (Download) is available for editing the code. Our software environment setup is nearly complete, now need to acquire the OpenAI API key.

Step 4. Get the OpenAI API key for FREE.

Please proceed to OpenAI’s website, log in, and select “Create new secret key” to obtain and copy the API key. Keep in mind that the entire API key cannot be viewed or copied later, so it is advisable to paste it into a Notepad file for future reference.

Afterward, visit platform.openai.com/account/usage to verify if you still have remaining credit. Should your free credit be depleted, it will be necessary to link a payment method to your OpenAI account.

Step 5. Train and Create an AI Chatbot With Custom Knowledge Base.

Add Your Documents to Train the AI Chatbot.

First, make a new folder named docs on your computer’s Desktop or another place you like. But remember to keep the folder name as docs.

After that, put the papers for learning in the “docs” folder. You can put in lots of text or PDF files, even ones you scanned. If you have a big table in Excel, you can turn it into a CSV or PDF file and then put it in the “docs” folder. You can also put in SQL database files, I haven’t tried many other kinds of files, but you can try it and see. For this article, I’m putting in one of my articles about NFTs in PDF format.

“ For a more efficient processing time, consider beginning with a smaller document (30-50 pages or files less than 100MB) to grasp the process, as larger documents will take longer to process based on your CPU and GPU, and will consume your OpenAI tokens rapidly.”

Step 6. Make the Code Ready.

Open a code editor, such as Sublime Text or Notepad++, and paste the code provided below. I received valuable assistance from armrrs on Google Colab, and I modified the code to ensure compatibility with PDF files and establish a Gradio interface.

from gpt_index import SimpleDirectoryReader, GPTListIndex, GPTSimpleVectorIndex, LLMPredictor, PromptHelperfrom langchain.chat_models import ChatOpenAIimport gradio as grimport sysimport os os.environ["OPENAI_API_KEY"] = 'Your API Key' def construct_index(directory_path): max_input_size = 4096 num_outputs = 512 max_chunk_overlap = 20 chunk_size_limit = 600 prompt_helper = PromptHelper(max_input_size, num_outputs, max_chunk_overlap, chunk_size_limit=chunk_size_limit) llm_predictor = LLMPredictor(llm=ChatOpenAI(temperature=0.7, model_name="gpt-3.5-turbo", max_tokens=num_outputs)) documents = SimpleDirectoryReader(directory_path).load_data() index = GPTSimpleVectorIndex(documents, llm_predictor=llm_predictor, prompt_helper=prompt_helper) index.save_to_disk('index.json') return index def chatbot(input_text): index = GPTSimpleVectorIndex.load_from_disk('index.json') response = index.query(input_text, response_mode="compact") return response.response iface = gr.Interface(fn=chatbot, inputs=gr.components.Textbox(lines=7, label="Enter your text"),outputs="text", title="Custom-trained AI Chatbot") index = construct_index("docs")iface.launch(share=True)

Afterward, navigate to the top menu, click on “File,” and choose “Save As…“. Subsequently, name the file app.py and switch the “Save as type” to “All types“.

Finally, save the file in the location where you initially created the “docs” folder (for instance, on the Desktop).

Ensure that the “docs” folder and “app.py” are both located in the same location, as in the screenshot. Note that the “app.py” file will be positioned outside the “docs” folder, not within it.

Return to the code in Notepad++ once more. Here, substitute “Your API Key” with the one you created earlier on OpenAI’s website.

Finally, press “Ctrl + S” to save the code. You are now ready to run the code.

Step 6. Create ChatGPT AI Bot with Custom Knowledge Base.

To begin, access the Terminal and run the below command to navigate to the Desktop, which is the location where I stored the “docs” folder and “app.py” file.

cd Desktop

Now run the command below.

python app.py

The document indexing process using the OpenAI LLM model will commence, with the duration contingent on the document size. Once completed, an “index.json” file will be generated on the Desktop.

If the Terminal displays no output, don’t fret, as it could still be processing the data. Note that processing a 30MB document typically takes approximately 10 seconds.

After the LLM has finished processing the data, you will discover a local URL. Please copy it for further use.

Paste the copied URL into the web browser, and your custom-trained ChatGPT-powered AI chatbot is ready; you can begin by asking the AI chatbot about the document’s content.

Ask more questions, and ChatGPT will reply based on your provided data, allowing you to build a customized AI chatbot. Train and create a chatbot with any information you prefer.

Step 7. Manage the custom AI chatbot.

Feel free to copy the public URL to share with friends and family; the link remains active for 72 hours. Ensure your computer stays on as the server instance is running locally.

To Stop the custom-trained AI chatbot, use “Ctrl + C” in the Terminal window. If that doesn’t work, try pressing “Ctrl + C” once more.

To restart the AI chatbot server again, go back to the Desktop and run the command below. Remember, the local web address stays the same, but the public web address changes every time you restart the server.

python app.py

If you wish to train the AI chatbot using new data, remove the files in the “docs” folder and insert new ones. You can include several files, but ensure they contain clear data for a consistent response.

Now run the code once more in the Terminal, and it will generate a new “index.json” file, automatically replacing the old one.

python app.py

Monitor your tokens by visiting OpenAI’s online dashboard and verifying the remaining free credit.

Finally, you don’t need to modify the code unless you wish to customize it further by changing the API key or the OpenAI model.

To sum it up, using ChatGPT API to train your AI chatbot with your own data helps make conversations fit your needs better. This guide highlights the importance of choosing and adjusting your data carefully. Keep learning, adapt to how users talk, and enjoy making your own special AI chatbot. Have fun building!.

Published by,

Felix tutors