[rdf4j-dev] LLM, ChatGPT and Llama

[Håvard Ottestad <hmottestad@xxxxxxxxx>]

Hi Everyone!

I’ve been experimenting with ChatGPT, the OpenAI GPT-4 API and also with various variants of llama.

How exactly RDF4J can work with a large language model is a bit of an open question. Personally I found GPT-4 to be very useful as a programming partner for generating tests, using libraries I’m unfamiliar with, discussing different approaches or architectures and as a general source of information. 

I’ve come to think that there are two sides to an integration between an LLM and RDF4J. 

The first, and most up in the air, is where an LLM would use RDF4J as a source of information when answering questions from a user. This could take the form of the LLM generating SPARQL queries, or through a list of various specialised helper methods. 

The second, and much simpler integration, is as a tool that the user or developer can call from within RDF4J. I can imagine that it could be useful to have a SPARQL function that the user can use to call an LLM. As an example a user could call llm(“GPT-4”, concat(“Summarise the following text: “, ?text)) and use the result as part of the query result or in an INSERT query.

What I would propose in the short term is a new Sail that provides this SPARQL function and an interface for writing different LLM backend connectors. That way users can connect to OpenAI if they want to use GPT or to local endpoint for models they can run themselves. For my own sake I’ve found Ollama to be an easy way to interact with llama models on my own computer. We are free to implement support for any backends we want. I’m sure various standards will evolve with time that would allow us to have a more generic backend that can connect to anything. 

I know that some commercial database vendors have already developed various ways to interact with OpenAI GPT, but that shouldn’t stop us from adding support in RDF4J. 

Anyone else started playing with LLMs?

Cheers,
Håvard