LLM SQL BaseDatosEspanol Mistral

🚀 Implement LLM and SQL to Interact with a Spanish Database in a Multimodal Approach

The challenge of generating accurate SQL for user queries has persisted for a long time, as it requires a deep understanding of both the query and the database. There are solutions that leverage the capabilities of large language models (LLMs) to generate SQL that aligns with the database schema and accurately retrieves the required data. However, there are several issues:

• These solutions are in English.
• There are some necessary knowledge elements that are not explicitly included in the database schema nor have been learned by the LLMs.

Therefore, a tool is proposed that can interact with a database in Spanish using a multimodal approach. As the first step, a custom dataset was created, and then it was trained with the support of PEFT (parameter - efficient fine - tuning). Through voice commands, instructions can be generated, and the corresponding responses can deliver results in the form of voice, text, or images depending on the nature of the response.

✨ Features

Model Details

In this project, fine - tuning was performed on two different models:

• Mistral 7B
• Code Llama To see which one would yield better results as a base.

The methodology is as follows: First, generate a dataset from data that includes questions and their corresponding SQL. The second step is to augment the data using different techniques. Once the data is generated, proceed to the training phase. After training the model, move on to the testing phase. Input instructions either by voice or text. In case of inconsistencies, proceed to the correction phase and obtain the expected results in a multimodal way. If the response is short, the results will be given by voice, and if they are very long, they will be presented in the form of tables or images.

Notebook

Here is a link to the GitHub with the training notebooks for each model. First, fine - tuning was performed on the Code Llama model to see the results.

• CodeLlama

As the second model, Mistral 7B was used. It was fine - tuned with the model quantized to 4 bits so that it can be run on more devices.

• Mistral

Dataset

Link to the Dataset

Models/Adapter

Link to the adapter of CodeLLama

Demo

Link to the demo

Results

After applying the tool, favorable statements were delivered for 80% of the data. After applying auto - correction, the figure reached 87%.

Contributions

Our contributions are listed as follows:

• Generate queries to a custom database via audio or text.
• Train a model with Spanish text to perform SQL queries.
• Deliver responses to queries using a multimodal approach (text, audio, or images).
• Validate the quality of responses with auto - correction mechanisms.

Team Members

• Asis Heriberto Rangel Ortiz
• Emmanuel Alexis Zarazua Rubio
• Néstor Emmanuel Briones Ramirez
• Ventura Rivera Robles
• Fernando Morales Flores

📄 License

This project is licensed under the Apache 2.0 license.