Interact with Your Own Data Using Ollama Hosted Phi3

 Phi3 is a very small large language model, with only 4 billion parameters compared to ChatGPT's 175 billion. Despite its smaller size, Phi3 offers a unique advantage: it can directly interact with your data without needing an API. In contrast, ChatGPT requires an API, and the free access to this API expires after three months unless you upgrade to a paid plan at $25 per month.

One of the main benefits of Ollama hosted Phi3 is that it doesn't require internet access and is completely free. However, running Phi3 locally does have its drawbacks. For optimal performance, it requires a high-spec PC or Mac, particularly one with a powerful GPU. While it can run on lower-spec machines, the performance will be significantly slower.

To demonstrate how Phi3 works, I have prepared a simple program that uploads a text file, processes it, and allows you to ask questions about the data. For this example, I used data about the national anthem of the Philippines. When I asked the untrained Phi3 who wrote the national anthem, it provided an incorrect answer. This is due to the limited amount of training data available for Phi3, as mentioned earlier. The following picture shows the reply of Phi3(without embeddings) from the DOS prompt:

I obtained the data from Wikipedia by just copying a few paragraphs and pasting it to Notepad and saving it as .txt file.

Here is the program and some explanations:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81

from langchain_chroma import Chroma from langchain_community.document_loaders import TextLoader from langchain_community.embeddings.sentence_transformer import ( SentenceTransformerEmbeddings, ) from langchain_text_splitters import CharacterTextSplitter from langchain_community.embeddings import OllamaEmbeddings from langchain_text_splitters import RecursiveCharacterTextSplitter from langchain_community.vectorstores import Chroma from langchain.prompts import ChatPromptTemplate, PromptTemplate from langchain_core.output_parsers import StrOutputParser from langchain_community.chat_models import ChatOllama from langchain_core.runnables import RunnablePassthrough from langchain.retrievers.multi_query import MultiQueryRetriever loader = TextLoader("lupanghinirang.txt") documents = loader.load() # Split the document into chunks text_splitter = CharacterTextSplitter(chunk_size=1000, chunk_overlap=0) docs = text_splitter.split_documents(documents) # Create the open-source embedding function embedding_function = SentenceTransformerEmbeddings(model_name="all-MiniLM-L6-v2") # Load it into Chroma db = Chroma.from_documents(docs, embedding_function) # Query the database query = "What is the National Anthem of the Philippines?" docs = db.similarity_search(query) # Split and chunk the documents text_splitter = RecursiveCharacterTextSplitter(chunk_size=7500, chunk_overlap=100) chunks = text_splitter.split_documents(docs) # Add to vector database vector_db = Chroma.from_documents( documents=chunks, embedding=OllamaEmbeddings(model="nomic-embed-text", show_progress=True), collection_name="local-rag" ) # LLM from Ollama local_model = "phi3" llm = ChatOllama(model=local_model) QUERY_PROMPT = PromptTemplate( input_variables=["question"], template="""You are an AI language model assistant. Your task is to generate five different versions of the given user question to retrieve relevant documents from a vector database. By generating multiple perspectives on the user question, your goal is to help the user overcome some of the limitations of the distance-based similarity search. Provide these alternative questions separated by newlines. Original question: {question}""", ) retriever = MultiQueryRetriever.from_llm( vector_db.as_retriever(), llm, prompt=QUERY_PROMPT ) # RAG prompt template = """Answer the question based ONLY on the following context: {context} Question: {question} """ prompt = ChatPromptTemplate.from_template(template) chain = ( {"context": retriever, "question": RunnablePassthrough()} | prompt | llm | StrOutputParser() ) chain.invoke("who composed the national anthem of the Philippines?")

Explanation

1. Loading and Splitting Documents:

   • The program starts by loading a text file (lupanghinirang.txt) using TextLoader.
   • The text is then split into chunks using CharacterTextSplitter.

2. Embedding and Storing Data:

   • An embedding function is created using the SentenceTransformerEmbeddings model.
   • The split documents are embedded and stored in a Chroma database.

3. Querying the Database:

   • A query is made to the database to find relevant documents.
   • The retrieved documents are further split into chunks using RecursiveCharacterTextSplitter.

4. Adding to Vector Database:

   • The chunks are embedded using OllamaEmbeddings and stored in another Chroma database.

5. Setting Up the Language Model:

   • The ChatOllama model (Phi3) is initialized.
   • A prompt template is created to generate multiple versions of a user query.

6. Retrieving Relevant Documents:

   • MultiQueryRetriever is used to retrieve relevant documents from the vector database.

7. Answering the Query:

   • A final prompt template is set up to answer the question based on the retrieved context.
   • The chain is executed to generate the answer to the question.

Here is the result when running the program:

This program demonstrates the capabilities of Phi3 in processing and interacting with local data. Despite its smaller size compared to ChatGPT, Phi3 can be a powerful tool for specific applications where internet access is limited or data privacy is a concern.

Upgrading the Ollama Hosted Phi3 Offline Chat Program

 In my previous post, "Chat with Ollama Phi3 in Python Offline," I introduced a basic program that allowed you to chat with Ollama's Phi3 language model offline using Python. Today, I’m excited to share an upgraded version of this program. This new version incorporates a graphical user interface (GUI) using PyQt6 and includes a text-to-speech (TTS) engine to read aloud Phi3's responses. The chat logs are displayed in a QTextEdit widget, making the interaction more user-friendly and accessible. This program is created in Python 3.10 same experience I encountered when I first create a chat program using Langchain and Openai. also a good pc/mac will deliver a better experience but it will still run on a Intel M3-100y but it will be very slow considering that I have downloaded the phi3 4b  quantized version.

Upgraded Features

1. Graphical User Interface (GUI): A chat window created using PyQt6.
2. Text-to-Speech: Utilizes pyttsx3 to read aloud Phi3's responses.
3. Chat Logs: Displays the conversation history in a QTextEdit widget.

Code Explanation

Below is the complete code for the upgraded program, followed by detailed explanations of each part.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80

import pyttsx3 from PyQt6.QtGui import * from PyQt6.QtCore import * from PyQt6.QtWidgets import * from langchain_community.llms import Ollama class Window(QMainWindow): def __init__(self): super(Window, self).__init__() # Set up the layout layout = QVBoxLayout() hlayout = QHBoxLayout() central_widget = QWidget() central_widget.setLayout(layout) self.setCentralWidget(central_widget) # Initialize the language model self.llm = Ollama(model="phi3") # Add widgets to the layout layout.addLayout(hlayout) self.te = QLineEdit(self) self.le = QTextEdit(self) self.le.setReadOnly(True) self.btnStart = QPushButton("Ask", self) hlayout.addWidget(self.te) hlayout.addWidget(self.btnStart) layout.addWidget(self.le) # Connect button click to function self.btnStart.clicked.connect(self.onClickedStart) # Set window properties self.setGeometry(25, 45, 900, 500) self.setWindowTitle('Speak') def onClickedStart(self): global text # Get user input and display it in the chat log user_input = self.te.text() self.le.append(f'You: {user_input}') # Get response from the language model response = self.llm.invoke(user_input) # Display the response in the chat log self.le.append(f'Phi3: {response}') # Prepare text for text-to-speech text = response self.worker = WorkerThread() self.worker.start() text = '' engine = [] error = '' i = 0 class WorkerThread(QThread): def run(self): global text, engine, error, i try: i += 1 engine.append(pyttsx3.init()) engine[i-1].setProperty('rate', 120) engine[i-1].say(text) engine[i-1].startLoop() except Exception as err: error = str(err) if __name__ == "__main__": import sys app = QApplication(sys.argv) w = Window() w.show() sys.exit(app.exec())

Detailed Explanation

Imports

• pyttsx3: A text-to-speech conversion library in Python.
• PyQt6: A set of Python bindings for the Qt application framework, used to create the GUI.
• Ollama: A module from the langchain_community package for interacting with the Phi3 language model.

Window Class

• Initialization (__init__ method):

  • Sets up the main layout and widgets (QLineEdit, QTextEdit, QPushButton).
  • Initializes the Phi3 language model using Ollama.
  • Connects the "Ask" button to the onClickedStart method.
  • Configures the window size and title.

• onClickedStart Method:

  • Gets the user's input from the QLineEdit widget.
  • Appends the user's input to the QTextEdit widget for display.
  • Sends the input to the Phi3 language model and gets the response.
  • Appends the response to the QTextEdit widget.
  • Initiates a text-to-speech conversion of the response using the WorkerThread class.

WorkerThread Class

• A QThread subclass that handles the text-to-speech functionality.
• run Method:
  • Initializes a new pyttsx3 engine for each response.
  • Sets the speech rate and converts the response text to speech.
  • Starts the speech engine loop to read the response aloud.

Main Block

• Creates a QApplication instance and a Window instance.
• Displays the Window and starts the event loop.

Conclusion

This upgraded version of the Ollama Phi3 offline chat program provides a more interactive and user-friendly experience. The integration of a GUI and text-to-speech functionality makes it easier to interact with the Phi3 model without needing to modify the underlying code. Give it a try and see how these enhancements improve your interactions with Phi3!

Separating Data Points

 Separating data points is a fundamental goal in many machine learning and data science tasks, particularly in classification problems. Here are several reasons why separating data points is important:

1. Classification

• Goal: The primary objective in classification problems is to assign labels to data points based on their features.
• Separation: By separating data points of different classes, a model can make accurate predictions about the class of new, unseen data points. Effective separation leads to higher classification accuracy.

2. Reducing Error

• Minimizing Misclassification: Separating data points helps to minimize the number of misclassified instances, thereby improving the overall performance of the model.
• Error Metrics: Common metrics such as accuracy, precision, recall, and F1-score are all improved when data points are well-separated according to their respective classes.

3. Improving Generalization

• Overfitting and Underfitting: A well-separated dataset helps in creating models that generalize better to new data. Models that fail to separate data points effectively might overfit (learn noise) or underfit (fail to capture the underlying trend).
• Decision Boundaries: Clear separation helps in defining decision boundaries that work well on both training data and unseen test data.

4. Interpretability

• Understanding the Model: When data points are well-separated, it is easier to understand and interpret the model’s decisions. This is especially useful in fields where interpretability is crucial, such as medical diagnostics or finance.
• Visualization: In lower dimensions (2D or 3D), well-separated data points can be visualized more clearly, helping stakeholders to understand the model's behavior.

5. Performance of Algorithms

• Algorithm Efficiency: Some machine learning algorithms, like Support Vector Machines (SVMs), work by finding the optimal separation between classes. The performance of these algorithms is directly linked to how well the data points can be separated.
• Convergence: Well-separated data points can lead to faster convergence in training algorithms, making the model training process more efficient.

6. Clustering and Anomaly Detection

• Clustering: In unsupervised learning, separating data points into distinct clusters helps in understanding the natural grouping within the data, which can be useful for exploratory data analysis.
• Anomaly Detection: Separating normal data points from anomalies helps in identifying outliers, which is critical in fields such as fraud detection, network security, and quality control.

7. Noise Reduction

• Handling Noisy Data: Separation helps in distinguishing between signal and noise. Effective separation techniques can help in identifying and mitigating the impact of noisy data points, leading to more robust models.

Example: Support Vector Machines (SVMs)

Support Vector Machines (SVMs) aim to find the hyperplane that best separates data points of different classes. This separation is achieved by maximizing the margin between the classes, which helps in improving the model’s robustness and accuracy. The use of kernel functions in SVMs allows for the separation of non-linearly separable data by projecting it into higher-dimensional spaces.

Conclusion

Separating data points is crucial for achieving high performance in various machine learning tasks. It leads to more accurate, interpretable, and generalizable models. Whether through classification, clustering, or anomaly detection, effective separation of data points underpins the success of many data science applications.

Projecting two-dimensional data into an infinite-dimensional space

 Projecting two-dimensional data into an infinite-dimensional space is a concept used in data science, particularly in the context of kernel methods. Here are some reasons why this might be desirable:

1. Handling Non-Linearly Separable Data

In many real-world problems, data points are not linearly separable in their original feature space. By projecting the data into a higher (potentially infinite) dimensional space, non-linear relationships can become linear. This allows for the application of linear algorithms in the new space, which can be more efficient and easier to implement.

2. Utilizing the Kernel Trick

The kernel trick is a technique that allows us to compute the dot product in the high-dimensional space without explicitly computing the transformation. Instead of mapping the data to the high-dimensional space and then computing the dot product, the kernel trick computes the dot product directly using a kernel function. This makes the computation feasible even if the high-dimensional space is infinite.

3. Improved Model Performance

Mapping data to a higher-dimensional space can improve the performance of certain algorithms, such as Support Vector Machines (SVMs). In this new space, it becomes easier to find a hyperplane that separates the data points of different classes. This often results in better classification accuracy.

4. Capturing Complex Relationships

High-dimensional projections can capture complex relationships and interactions between features that are not apparent in the original feature space. This allows for more expressive models that can better fit the data.

5. Flexibility in Choice of Kernel

Different kernel functions (such as polynomial, radial basis function (RBF), and sigmoid kernels) correspond to different ways of projecting data into higher-dimensional spaces. This flexibility allows practitioners to choose a kernel that best captures the underlying structure of the data for a given problem.

Example: The Radial Basis Function (RBF) Kernel

The RBF kernel is commonly used in SVMs and other kernel-based methods. It effectively projects the data into an infinite-dimensional space, enabling the separation of data points that are not linearly separable in the original space. The RBF kernel is defined as:

Here, $x$ and $x'$ are data points, $\|\cdot\|$ denotes the Euclidean distance, and $\sigma$ is a parameter that defines the width of the Gaussian function. The RBF kernel measures similarity in a way that considers all possible dimensions, effectively projecting the data into an infinite-dimensional space.

Conclusion

Projecting two-dimensional (or low-dimensional) data into an infinite-dimensional space through the use of kernels allows for the handling of complex, non-linear relationships in a computationally efficient manner. This is a powerful technique in machine learning, enabling the development of more accurate and robust models.

Run a Simple PyGame program to any Browser

 Pygbag is a project that allows you to run Pygame-based Python games in the browser. It converts your Pygame projects into WebAssembly, enabling them to run in modern web browsers. Here’s a step-by-step guide on how to create a simple Pygame project and use Pygbag to run it in the browser:

Step-by-Step Guide to Creating a Pygame Project with Pygbag

1. Install Pygame:

    Make sure you have Pygame installed on your system.

pip install pygame

2.  Create a Simple Pygame Project:

     Create a directory for your project and add a simple Pygame script.

    Here’s an example main.py script:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48

import asyncio import pygame import sys # Initialize Pygame pygame.init() # Set up the display size = width, height = 800, 600 screen = pygame.display.set_mode(size) pygame.display.set_caption("Pygbag Example") # Set up colors black = (0, 0, 0) blue = (0, 0, 255) # Set up the ball ball = pygame.image.load("ball.png") ballrect = ball.get_rect() # Set up ball speed speed = [2, 2] async def main(): global ballrect # Explicitly declare ballrect as global to modify it within the function while True: for event in pygame.event.get(): if event.type == pygame.QUIT: pygame.quit() sys.exit() elif event.type == pygame.KEYDOWN: if event.key == pygame.K_ESCAPE: pygame.quit() sys.exit() ballrect = ballrect.move(speed) if ballrect.left < 0 or ballrect.right > width: speed[0] = -speed[0] if ballrect.top < 0 or ballrect.bottom > height: speed[1] = -speed[1] screen.fill(black) screen.blit(ball, ballrect) pygame.display.update() await asyncio.sleep(0) # Yield control to the event loop asyncio.run(main())

    Make sure you have an image named ball.png in the same directory as your main.py script.

3.  Install Pygbag:

    Install Pygbag using pip.

pip install pygbag

4. Run Pygbag to Package Your Game:

    Use Pygbag to package your game and run it in the browser.

pygbag <my_pygame_project>

5.  Open the Game in Your Browser:

    After running the above command, Pygbag will start a local web server and provide a URL(http://localhost:8000/). Open this URL in your web browser to play your game.

By following these steps, you can create a simple Pygame project and use Pygbag to run it in any web browser. This allows you to leverage Pygame’s capabilities while making your game accessible on the web.

Chat with Ollama Phi3 in Python Offline

 Ollama lets you download and run local versions of LLMs of your choice. In this article, I downloaded Phi3 with 8B parameters.

To begin, download the Ollama software from their website. Then, in the DOS prompt, type ollama run phi3. The download will begin, and after downloading, it will start Phi3 right away so you can chat with Phi3 immediately.

Install the required library.

pip install langchain_community

Next, I prepared this Python program to interact with Phi3 locally, i.e., offline:

1 2 3 4 5 6 7 8

from langchain_community.llms import Ollama llm = Ollama( model="phi3" ) # assuming you have Ollama installed and have the phi3 model pulled with `ollama pull phi3` x = llm.invoke("Tell me a joke") print(x)

A Simple Python Email Server for offline use

 Creating a simple offline Python email server can be achieved using the smtplib library for sending emails and a basic SMTP server for testing. This server won't connect to the internet but will allow you to send and receive emails locally.

Here’s a step-by-step guide to set up a simple offline Python email server:

1. Install Python Libraries: Ensure you have Python installed. You may also need the smtplib and email libraries, which are typically included in the standard library.

2. Create a Local SMTP Server: Use Python’s smtpd module to create a local SMTP server.

3. Send Emails: Use the smtplib module to send emails through the local server.

Here’s an example implementation:

Step 1: Create a Local SMTP Server

Create a Python script (local_smtp_server.py) to start a local SMTP server:

1 2 3 4 5 6 7 8 9 10 11 12 13 14

import smtpd import asyncore class CustomSMTPServer(smtpd.SMTPServer): def process_message(self, peer, mailfrom, rcpttos, data, **kwargs): print('Message received from:', peer) print('Message addressed from:', mailfrom) print('Message addressed to :', rcpttos) print('Message length :', len(data)) return if __name__ == '__main__': server = CustomSMTPServer(('localhost', 1025), None) asyncore.loop()

Step 2: Run the Local SMTP Server

Run the script in your terminal:

1	python local_smtp_server.py

The server will start and listen for incoming emails on localhost:1025.

Step 3: Send Emails Using the Local SMTP Server

Create another Python script (send_email.py) to send emails using the local server:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19

import smtplib from email.mime.text import MIMEText def send_email(subject, body, from_email, to_email): msg = MIMEText(body) msg['Subject'] = subject msg['From'] = from_email msg['To'] = to_email with smtplib.SMTP('localhost', 1025) as server: server.sendmail(from_email, [to_email], msg.as_string()) if __name__ == '__main__': subject = 'Test Email' body = 'This is a test email sent from a local SMTP server.' from_email = 'sender@example.com' to_email = 'receiver@example.com' send_email(subject, body, from_email, to_email)

Step 4: Run the Email Sending Script

Run the script in your terminal:

1	python send_email.py

You should see the email details printed in the terminal where the local SMTP server is running, indicating that the email was successfully sent and received by the server.

This setup allows you to test email sending and receiving functionalities locally without the need for an internet connection.