This post involves building question and answering system to better understand the behaviour and prediction of stocks and crypto currency. The important components to building this system are the following

1. Open Source Large Language Models such as Llama-2
2. Open AI (GPT turbo 3.5 or GPT-4.0)
3. Financial data from Yahoo Finance and finnhub
4. Modal for training large language models
5. Transformer library from HuggingFace

The following diagram demonstrates end-to-end process.

There are 3 main parts to finetuning which are decribed below.

Data creation:

The data creation module consists of 3 parts

1. Yahoo Finance:

One of the most valuable resources for monitoring the daily fluctuations in stock prices is Yahoo Finance. This platform offers comprehensive data, including daily, weekly, and monthly insights into stock prices. The yfinance API in Python serves as a powerful tool, providing useful information related to stock prices. I extract weekly information of the stock prices and categories the weekly movement in stocks as either going Up or Down along with %.

2. Finnhub:

Finnhub.io offers a Python-based API that facilitates the extraction of information pertinent to stocks / cryptocurrency. Leveraging Finnhub, I retrieve all news articles associated with a specified stock or cryptocurrency.

3. GPT-turbo 3.5 or GPT-4.0

The objective is to generate question and answering pairs for training the Llama-2 model. Using OpenAI’s API, I supply information encompassing all the news associated with a specific stock, including its trend (either upward or downward). Then I ask GPT-4 to provide pros and cons about the stock based on the news information and provide analysis on why the stock might go up or down. Following is the code snippet to call open ai’s API

  
  completion = openai_client.chat.completions.create(
                        model="gpt-3.5-turbo",
                        messages=[
                            {"role": "system", "content": system_prompt},
                            {"role": "user", "content": prompt}
                          ]
                    )
  #system prompt: Telling GPT what is the role.
  #user prompt: Information related to the stock / cryptocurrency
  

Finetunning:

Once the data is generated for question and answering its time to train llama-2. Meta provides different llama-2 models, I am using llama-2 7B model from huggingface. Rather than finetuning all the weights of llama-2, I use LoRA (Low-Rank Adaptation) technique to fine tune llama-2. LoRA significantly reduces the number of trainable parameters by inserting smaller number of new weights into the model and training on them. Its space and memory efficient. Huggingface provides all the required API’s to train llama-2 using LoRA. Following is the LoRA config used for training

  
  peft_config = LoraConfig(
      task_type=TaskType.CAUSAL_LM,
      inference_mode=False,
      r=8,
      lora_alpha=16,
      lora_dropout=0.1,
      bias='none',
  )

  model = get_peft_model(model, peft_config)
  

Training on the cloud using Modal

Modal offers a convenient solution for executing code in the cloud. Notably, it provides a swift and hassle-free approach to training extensive language models, eliminating concerns about GPU memory limitations. All that is required is to specify the necessary libraries/packages for code execution. In essence, Modal initiates containers in the cloud, providing a seamless method for training large language models. Following are the basic steps

  
# step 1: Create a container image
llm_train_image=(modal.Image.debian_slim().pip_install(
        "datasets",
        "matplotlib",
        "scikit-learn",
        "torch",
        "accelerate==0.21.0",
        "peft==0.4.0",
        "bitsandbytes==0.40.2",
        "transformers==4.31.0",
        "trl==0.4.7"        
    )
    .run_function(download_models)
    .pip_install("wandb==0.15.0")
    .pip_install("rouge-score")
    .pip_install("scikit-learn")
    ) 

# step 2: Create a stub that uses the image
stub = modal.Stub("train_llm", image=llm_train_image)

# step 3: Add this decorator on top of the function, all the code inside can use the packages from the image
@stub.function(mounts=[modal.Mount.from_local_dir("/home/suhaspillai/Suhas/git/llms/ask-fsdl/fin_llm/src/",
 remote_path="/root/")], gpu="A100", timeout=30000, network_file_systems={model_store_path: vol})
def load_dataset():
  --- Training code ---
  

Once training is finished I plot training and validation loss for the trained model trained on A-100 GPUs for 100 epochs.

Once the model is trained, we can provide it with the news articles for a specific week and it would provide pros, cons, prediction and analysis about the stocks and cryptocurrency. Following is the output from llama-2 and FT-llama-2.

Code for the project can be found on github