Monday, December 4, 2023

Fine-Tuning a Language Model(LLAMA 2) with GradientAI

Fine-tuning a language model allows you to adapt a pre-existing model to your specific needs. In this example, we'll demonstrate how to fine-tune a language model using the GradientAI platform. This Python program leverages the GradientAI SDK to interact with the platform's services.

Prerequisites

Before running the program, ensure you have the GradientAI SDK installed. You can install it using:

pip install gradientai

Program Overview

This Python program fine-tunes a language model based on a selected base model from the GradientAI platform. The base model used here is named "nous-hermes2". The fine-tuned model is then used to generate responses to given queries.

Code Walkthrough

Let's go through the key components of the program:

1. Setting up Gradient Credentials:

os.environ['GRADIENT_ACCESS_TOKEN'] = "xx"

os.environ['GRADIENT_WORKSPACE_ID'] = "yy"

Replace "xx" and "yy" with your Gradient access token and workspace ID.

2. Initializing Gradient:

with Gradient() as gradient:

This block initializes the Gradient SDK.

3. Creating a Model Adapter:

base_model = gradient.get_base_model(base_model_slug="nous-hermes2")

new_model_adapter = base_model.create_model_adapter(name="test model 3")

It creates a new model adapter based on the chosen base model.

4. Generating Text (Before Fine-Tuning):

completion = new_model_adapter.complete(query=sample_query, max_generated_token_count=100).generated_output

Generates text based on a sample query using the model before fine-tuning.

5. Fine-Tuning the Model:

new_model_adapter.fine_tune(samples=samples)

Fine-tunes the model using a set of samples (input and response pairs).

6. Generating Text (After Fine-Tuning):

completion = new_model_adapter.complete(query=sample_query, max_generated_token_count=100).generated_output

Generates text based on a sample query using the fine-tuned model.

7. Cleaning Up:

new_model_adapter.delete()

Running the Program

Simply run the Python script, and it will output the generated text before and after fine-tuning. Adjust the number of fine-tuning epochs (num_epochs) based on your requirements.

Feel free to customize the samples provided in the program to better suit your specific use case or domain.

Here is the complete python program :

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
from gradientai import Gradient

import os
os.environ['GRADIENT_ACCESS_TOKEN'] = "xx"
os.environ['GRADIENT_WORKSPACE_ID'] = "yy" 
def main():
  with Gradient() as gradient:
      base_model = gradient.get_base_model(base_model_slug="nous-hermes2")

      new_model_adapter = base_model.create_model_adapter(
          name="test model 3"
      )
      print(f"Created model adapter with id {new_model_adapter.id}")
      sample_query = "### What is a for loop in Python? \n\n### Response:"
      print(f"Asking: {sample_query}")

      # before fine-tuning
      completion = new_model_adapter.complete(query=sample_query, max_generated_token_count=100).generated_output
      print(f"Generated (before fine-tune): {completion}")

      samples = [
        {"inputs": "### Instruction: What is a variable in Python? \n\n### Response: In Python, a variable is used to store data values."},
        {"inputs": "### Instruction: Explain the 'if' statement in Python. \n\n### Response: The 'if' statement is used for conditional execution in Python."},
        {"inputs": "### Instruction: How do you define a function in Python? \n\n### Response: In Python, you can define a function using the 'def' keyword."},
        {"inputs": "### Instruction: What is a for loop in Python? \n\n### Response: A for loop in Python is used for iterating over a sequence."},
        {"inputs": "### Instruction: Explain the concept of list comprehension in Python. \n\n### Response: List comprehension is a concise way to create lists in Python."},
        {"inputs": "### Instruction: How do you handle exceptions in Python? \n\n### Response: Exception handling in Python is done using the 'try', 'except', and 'finally' blocks."},
        {"inputs": "### Instruction: What is the purpose of the 'import' statement in Python? \n\n### Response: The 'import' statement is used to include external modules in Python."},
        {"inputs": "### Instruction: How can you open and read a file in Python? \n\n### Response: You can use the 'open' function to open a file and 'read' method to read its contents in Python."},
        {"inputs": "### Instruction: What is the purpose of the 'else' statement in Python? \n\n### Response: The 'else' statement is used for code that should be executed if the preceding 'if' statement is false."},
        {"inputs": "### Instruction: Explain the concept of a dictionary in Python. \n\n### Response: In Python, a dictionary is an unordered collection of key-value pairs."},
        {"inputs": "### Instruction: How can you install external packages in Python? \n\n### Response: External packages in Python can be installed using the 'pip' tool."},
        {"inputs": "### Instruction: What is the role of the 'return' statement in a function? \n\n### Response: The 'return' statement in Python is used to exit a function and return a value."},
        {"inputs": "### Instruction: How do you create a virtual environment in Python? \n\n### Response: You can create a virtual environment using the 'venv' module or 'virtualenv' package."},
        {"inputs": "### Instruction: Explain the concept of object-oriented programming (OOP) in Python. \n\n### Response: Object-oriented programming in Python involves creating and using classes and objects."},
        {"inputs": "### Instruction: What is the difference between '==' and 'is' in Python? \n\n### Response: '==' is used for value equality, while 'is' is used for object identity."},
        {"inputs": "### Instruction: How do you use list slicing in Python? \n\n### Response: List slicing in Python is done using the syntax list[start:stop:step]."},
        {"inputs": "### Instruction: What is the purpose of 'try', 'except', and 'finally' in Python exception handling? \n\n### Response: 'try' is used for the main code block, 'except' catches exceptions, and 'finally' ensures code execution regardless of an exception."},
        {"inputs": "### Instruction: Explain the concept of a lambda function in Python. \n\n### Response: A lambda function in Python is an anonymous function created using the 'lambda' keyword."},
        {"inputs": "### Instruction: How can you open and read a file in Python? \n\n### Response: You can use the 'open()' function to open a file and various methods like 'read()' to read its contents."},
        {"inputs": "### Instruction: What is the purpose of the 'pass' statement in Python? \n\n### Response: The 'pass' statement is a null operation, used when syntax is required but no action is desired."},
        {"inputs": "### Instruction: Explain the difference between 'append()' and 'extend()' methods in Python lists. \n\n### Response: 'append()' adds a single element to the end of a list, while 'extend()' adds elements from an iterable."},
        {"inputs": "### Instruction: How do you handle user input in Python? \n\n### Response: User input in Python can be obtained using the 'input()' function."},
        {"inputs": "### Instruction: What is a decorator in Python? \n\n### Response: A decorator in Python is a design pattern that allows you to extend or modify the behavior of functions or methods."},
        {"inputs": "### Instruction: Explain the purpose of the '__init__' method in Python classes. \n\n### Response: The '__init__' method is a constructor in Python classes, used to initialize object attributes."},
        {"inputs": "### Instruction: How can you perform multi-threading in Python? \n\n### Response: Python supports multi-threading using the 'threading' module."},
        {"inputs": "### Instruction: Explain the use of 'self' in Python classes. \n\n### Response: 'self' is a convention in Python, representing the instance of the class and used to access instance variables."},
        {"inputs": "### Instruction: What is a Python decorator used for? \n\n### Response: A decorator is used to modify or extend the behavior of functions or methods without modifying their actual code."},
        {"inputs": "### Instruction: How can you handle exceptions in Python? \n\n### Response: Exceptions in Python can be handled using 'try', 'except', and 'finally' blocks."},
        {"inputs": "### Instruction: Explain the difference between '==', 'is', and 'in' operators in Python. \n\n### Response: '==' checks for equality, 'is' checks for identity, and 'in' checks for membership."},
        {"inputs": "### Instruction: What is the purpose of the 'with' statement in Python? \n\n### Response: The 'with' statement simplifies resource management by ensuring proper acquisition and release of resources."},
        {"inputs": "### Instruction: How do you create a virtual environment in Python? \n\n### Response: You can create a virtual environment using the 'venv' module or 'virtualenv' package."},
        {"inputs": "### Instruction: Explain the concept of list comprehension in Python. \n\n### Response: List comprehension is a concise way to create lists in Python using a single line of code."},
        {"inputs": "### Instruction: What is the purpose of the 'map()' function in Python? \n\n### Response: 'map()' applies a function to all items in an input list and returns an iterator."},
        {"inputs": "### Instruction: How can you convert a string to lowercase in Python? \n\n### Response: You can use the 'lower()' method to convert a string to lowercase."},
        {"inputs": "### Instruction: Explain the Global Interpreter Lock (GIL) in Python. \n\n### Response: The GIL is a mutex that protects access to Python objects, preventing multiple threads from executing Python bytecodes at once."},
    
    ]

      # this is where fine-tuning happens
      # num_epochs is the number of times you fine-tune the model
      # more epochs tends to get better results, but you also run the risk of "overfitting"
      # play around with this number to find what works best for you
      num_epochs = 3
      count = 0
      while count < num_epochs:
          print(f"Fine-tuning the model, iteration {count + 1}")
          new_model_adapter.fine_tune(samples=samples)
          count = count + 1

      # after fine-tuning
      completion = new_model_adapter.complete(query=sample_query, max_generated_token_count=100).generated_output
      print(f"Generated (after fine-tune): {completion}")

      new_model_adapter.delete()

if __name__ == "__main__":
    main()

Here is the source where this info came from:


This is a little bit more complicated that using langchain in terms of data preparation.
By at
Labels: , ,

No comments:

Post a Comment

Subscribe to: Post Comments (Atom)