This reposotory stores an implementation of the LLMA Dialogue Model for experiment purpose. The paper of LLMA Dialogue Model can be found here.
The implementation is a Java project using Maven and MELT.
This project uses GPT through Azure OpenAI. You will need to update model information in OpenAI.java file. The recommended model is gpt-4-32k.
Implementation is stored in simpleSealsMatcher folder. A copy of the used OAEI dataset Anatomy is stored in the .*/java/DataSet/ folder.
Results we retrieved and reported in paper is in the results folder. Note, we didn't use MELT generated statics, which you may find in the performance csv, as we found problems the toolkit handling our data. For example, one reference alignment between "http://human.owl#NCI_C12499" and "http://mouse.owl#MA_0000237", our exact alignment was evaluated as "false positive".
Note: This project is started for experiments. Though we tried to clean codes, you may still find testing purpose codes.
The project entrance is in Main class, and other class work as following:
- MyMatcher is the matcher implemented using MELT. It's used as the entrance of the model.
- OntologyAgent is the agent class negotiating in the model. It contains functions agents need to negotiate and get correspondence.
- OpenAI is responsible for all API calls and prompt assembly that agents would need.
- Weaviate is the vector database used by agent. For Weaviate instruction, see here
The implementation LLMA paper used is the v0.1.0
info += ontClass.getLocalName() +"\n";
info += ontClass.getLabel(null) +"\n";
info += ontClass.getComment(null);json_row.put("vector", ai.getEmbeddings(info));
json_row.put("uri", ontClass.getURI());
json_row.put("isNegotiated", false);The prompt template of selecting the most likely entity for alignment:
<Problem Definition>
In this task, we are giving a) one subject entity, and b) a set of entities for potential alignment in the form of Relation(Subject, EntitiesOfOtherAgent), which consist of URIs and labels.
<Subject Entity>
<Entities Of Other Agent>
Among all entities of other ontology, select all entities that you think having a possibility aligning with the subject entity? Please only answer with the index of entity (just the index, for example "1, 2, 4"). Answer "no" if you think none of them aligns with the subject entity.
The prompt template of deciding if two entities are potentially aligned:
<Problem Definition>
In this task, we are given two entities in the form of Relation(Subject, Object), which consist of URIs and labels.
<Entity Triples>
[Entity 1:Entity2]
Do you think these two entities are aligned? If so, please output:yes, otherwise, please output:no(just "yes" or "no", small character no other symbols required)
The information of entity:
URI: <getURI()>
Label: <getLabel(null)>
Local name: <getLocalName>
Comment: <getComment>
<if(listProperty startsWith(http) and not null), then below>
Property: <getPredicate().getLocalName()>
Value: <getObject().toString()>
... More listProperties...
<sub-info if exists, then below>
================ Relevant entity of this ontology ================
Label: <getLabel(null)>
URI: <getURI()>
Local name: <getLocalName()>
Comment: <getComment(null)>
<if(listProperty startsWith(http) and not null), then below>
Property: <getPredicate().getLocalName()>
Value: <getObject().toString()>
... More listProperties...
Shiyao Zhang, Yuji Dong, Yichuan Zhang, Terry R. Payne, and Jie Zhang. 2024. Large Language Model Assissted Multi-Agent Dialogue for Ontology Alignment. In Proceedings of the 23rd International Conference on Autonomous Agents and Multiagent Systems (AAMAS '24). International Foundation for Autonomous Agents and Multiagent Systems, Richland, SC, 2594–2596.