An Easy-to-use Knowledge Editing Framework for LLMs.
An Easy-to-use Knowledge Editing Framework for Large Language Models.
Installation • QuickStart • Doc • Paper • Demo • Benchmark • Contributors • Slides • Video • Featured By AK
Accelerate
.
This repository is a subproject of KnowLM.
A Comprehensive Study of Knowledge Editing for Large Language Models [paper][benchmark][code]
AAAI 2024 Tutorial [Google Drive] [Baidu Pan]
AACL 2023 Tutorial [Google Drive] [Baidu Pan]
There is a demonstration of editing. The GIF file is created by Terminalizer.
Deployed models may still make unpredictable errors. For example, Large Language Models (LLMs) notoriously hallucinate, perpetuate bias, and factually decay, so we should be able to adjust specific behaviors of pre-trained models.
Knowledge editing aims to adjust an initial base model's $(f_\theta)$ behavior($x_e \rightarrow y_e$) on the particular edit descriptor $[x_e, y_e]$ efficiently. There are usually three forms:
Inject knowledge that LLMs have not seen before. such as:
LLMs often suffer from knowledge cutoff issue, EasyEdit can update outdated knowledge. such as:
EasyEdit can erase sensitive information. such as:
Without influencing the model behavior on unrelated samples, the ultimate goal is to create an edited model $(f_\theta')$.
Detoxifying LLM strives to build a safe and trustworthy large language model (LLM). Knowledge editing focuses on specific areas for permanent adjustment without compromising overall performance. Then, detoxifying LLM via knowledge editing leverages a small amount of data, usually an instance, to correct the toxic behaviors of the LLM. The edited LLM can defend against various malicious inputs. README
Editing Task for Image Captioning and Visual Question Answering. README
The proposed task takes the preliminary attempt to edit LLMs' personalities by editing their opinions on specific topics, given that an individual's opinions can reflect aspects of their personality traits. We draw upon the established BIG FIVE theory as a basis for constructing our dataset and assessing the LLMs' personality expressions. README
Evaluation
Logits-based
Generation-based
While for assessing Acc and TPEI, you can download the trained classifier from here.
The knowledge editing process generally impacts the predictions for a broad set of inputs that are closely associated with the edit example, called the editing scope.
A successful edit should adjust the model’s behavior within the editing scope while remaining unrelated inputs(as below formula).
$$ f_{\theta_{e}}(x) = \begin{cases} y_e & \text{if } x \in I(x_e,y_e) \ f_{\theta}(x) & \text{if } x \in O(x_e, y_e) \end{cases} $$
In addition to this, the performance of knowledge editing should be measured from multiple dimensions:
Reliability
: the success rate of editing with a given editing descriptionGeneralization
: the success rate of editing within the editing scopeLocality
: whether the model's output changes after editing for unrelated inputsPortability
: the success rate of editing for factual reasoning(one hop, synonym, one-to-one relation)Efficiency
: time and memory consumption required during the editing processEasyEdit is a Python package for edit Large Language Models (LLM) like GPT-J
, Llama
, GPT-NEO
, GPT2
, T5
(support models from 1B to 65B), the objective of which is to alter the behavior of LLMs efficiently within a specific domain without negatively impacting performance across other inputs. It is designed to be easy to use and easy to extend.
EasyEdit contains a unified framework for Editor, Method and Evaluate, respectively representing the editing scenario, editing technique, and evaluation method.
Each Knowledge Editing scenario comprises of three components:
Editor
: such as BaseEditor(Factual Knowledge and Generation Editor) for LM, MultiModalEditor(MultiModal Knowledge).Method
: the specific knowledge editing technique used(such as ROME, MEND, ..).Evaluate
: Metrics for evaluating knowledge editing performance.
Reliability
, Generalization
, Locality
, Portability
The current supported knowledge editing techniques are as follows:
Due to the limited compatibility of this toolkit and limited by the transformer version, some knowledge editing methods including T-Patcher, KE, CaliNet are not supported. Similarly, the MALMEN method is only partially supported due to the same reasons and will continue to be improved upon in the first half of the year.
You can choose different editing methods according to your specific needs.
Method | T5 | GPT-2 | GPT-J | GPT-NEO | LlaMA | Baichuan | ChatGLM2 | InternLM | Qwen | Mistral |
---|---|---|---|---|---|---|---|---|---|---|
FT | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
AdaLoRA | ✅ | |||||||||
SERAC | ✅ | ✅ | ✅ | ✅ | ||||||
IKE | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
MEND | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
KN | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | |
ROME | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | |
r-ROME | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | |
MEMIT | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | |
GRACE | ✅ | ✅ | ✅ | |||||||
MELO | ✅ | |||||||||
PMET | ✅ | ✅ | ||||||||
InstructEdit | ✅ | ✅ |
❗️❗️ EasyEdit supports editing ChatGPT with FT. An edit for
gpt-3.5-turbo
returns model_name(for example,ft: GPT-3.5-turbo-0613 :personal::7tWZkLzq
) instead model weights.
❗️❗️ If you intend to use Mistral, please update the
transformers
library to version 4.34.0 manually. You can use the following code:pip install transformers==4.34.0
.
Benchmark: KnowEdit [Hugging Face][WiseModel][ModelScope]
❗️❗️ To be noted, KnowEdit is constructed by re-organizing and extending exsiting datasests including WikiBio, ZsRE, WikiDataCounterfact, WikiDataRecent, convsent, Sanitation to make a comprehensive evaluation for knowledge editing. Special thanks to the builders and maintainers of the those datasets.
Please note that Counterfact and WikiDataCounterfact are not the same dataset.
Task | Knowledge Insertion | Knowledge Modification | Knowledge Erasure | |||
---|---|---|---|---|---|---|
Datasets | Wikirecent | ZsRE | WikiBio | WikiDatacounterfact | Convsent | Sanitation |
Type | Fact | Question Answering | Hallucination | Counterfact | Sentiment | Unwanted Info |
# Train | 570 | 10,000 | 592 | 1,455 | 14,390 | 80 |
# Test | 1,266 | 1230 | 1,392 | 885 | 800 | 80 |
We provide detailed scripts for user to easily use KnowEdit, please refer to examples.
knowedit
├── WikiBio
│ ├── wikibio-test-all.json
│ └── wikibio-train-all.json
├── ZsRE
│ └── ZsRE-test-all.json
├── wiki_counterfact
│ ├── test_cf.json
│ └── train_cf.json
├── convsent
│ ├── blender_test.json
│ ├── blender_train.json
│ └── blender_val.json
├── convsent
│ ├── trivia_qa_test.json
│ └── trivia_qa_train.json
└── wiki_recent
├── recent_test.json
└── recent_train.json
dataset | Google Drive | BaiduNetDisk | Description |
---|---|---|---|
ZsRE plus | [Google Drive] | [BaiduNetDisk] | Question Answering dataset using question rephrasings |
Counterfact plus | [Google Drive] | [BaiduNetDisk] | Counterfact dataset using Entity replacement |
We provide zsre and counterfact datasets to verify the effectiveness of knowledge editing. You can download them here. [Google Drive], [BaiduNetDisk].
editing-data
├── counterfact
│ ├── counterfact-edit.json
│ ├── counterfact-train.json
│ └── counterfact-val.json
├── locality
│ ├── Commonsense Task
│ │ ├── piqa_valid-labels.lst
│ │ └── piqa_valid.jsonl
│ ├── Distracting Neighbor
│ │ └── counterfact_distracting_neighbor.json
│ └── Other Attribution
│ └── counterfact_other_attribution.json
├── portability
│ ├── Inverse Relation
│ │ └── zsre_inverse_relation.json
│ ├── One Hop
│ │ ├── counterfact_portability_gpt4.json
│ │ └── zsre_mend_eval_portability_gpt4.json
│ └── Subject Replace
│ ├── counterfact_subject_replace.json
│ └── zsre_subject_replace.json
└── zsre
├── zsre_mend_eval.json
├── zsre_mend_train_10000.json
└── zsre_mend_train.json
spouse
dataset | Google Drive | HuggingFace Dataset | Description |
---|---|---|---|
ConceptEdit | [Google Drive] | [HuggingFace Dataset] | dataset for editing conceptual knowledge |
data
└──concept_data.json
├──final_gpt2_inter.json
├──final_gpt2_intra.json
├──final_gptj_inter.json
├──final_gptj_intra.json
├──final_llama2chat_inter.json
├──final_llama2chat_intra.json
├──final_mistral_inter.json
└──final_mistral_intra.json
Concept Specific Evaluation Metrics
Instance Change
: capturing the intricacies of these instance-level changesConcept Consistency
: the semantic similarity of generated concept definitiondataset | Google Drive | BaiduNetDisk | Description |
---|---|---|---|
E-IC | [Google Drive] | [BaiduNetDisk] | dataset for editing Image Captioning |
E-VQA | [Google Drive] | [BaiduNetDisk] | dataset for editing Visual Question Answering |
editing-data
├── caption
│ ├── caption_train_edit.json
│ └── caption_eval_edit.json
├── locality
│ ├── NQ dataset
│ │ ├── train.json
│ │ └── validation.json
├── multimodal_locality
│ ├── OK-VQA dataset
│ │ ├── okvqa_loc.json
└── vqa
├── vqa_train.json
└── vqa_eval.json
dataset | HuggingFace Dataset | Description |
---|---|---|
SafeEdit | [HuggingFace Dataset] | dataset for detoxifying LLMs |
data
└──SafeEdit_train.json
└──SafeEdit_val.json
└──SafeEdit_test.json
Detoxifying Specific Evaluation Metrics
Defense Duccess (DS)
: the detoxification success rate of edited LLM for adversarial input (attack prompt + harmful question), which is used to modify LLM.Defense Generalization (DG)
: the detoxification success rate of edited LLM for out-of-domain malicious inputs.General Performance
: the side effects for unrelated task performance.Method | Description | GPT-2 | LlaMA |
---|---|---|---|
IKE | In-Context Learning (ICL) Edit | [Colab-gpt2] | [Colab-llama] |
ROME | Locate-Then-Edit Neurons | [Colab-gpt2] | [Colab-llama] |
MEMIT | Locate-Then-Edit Neurons | [Colab-gpt2] | [Colab-llama] |
Note: Please use Python 3.9+ for EasyEdit To get started, simply install conda and run:
git clone https://github.com/zjunlp/EasyEdit.git
conda create -n EasyEdit python=3.9.7
...
pip install -r requirements.txt
We packaged the environment, you can download Docker from this link.
Pull the Docker image from Docker Hub or Aliyun:
docker pull zjunlp/easyedit
docker pull registry.cn-hangzhou.aliyuncs.com/zjunlp/easyedit:v1
If you want to build the Docker image locally, you can clone the project to your local machine and build the Docker image:
git clone https://github.com/zjunlp/EasyEdit.git
cd EasyEdit
docker build -t your-image-name .
Then run the Docker image as a container:
docker run -p 8080:80 your-image-name
Our results are all based on the default configuration
llama-2-7B | chatglm2 | gpt-j-6b | gpt-xl | |
---|---|---|---|---|
FT | 60GB | 58GB | 55GB | 7GB |
SERAC | 42GB | 32GB | 31GB | 10GB |
IKE | 52GB | 38GB | 38GB | 10GB |
MEND | 46GB | 37GB | 37GB | 13GB |
KN | 42GB | 39GB | 40GB | 12GB |
ROME | 31GB | 29GB | 27GB | 10GB |
MEMIT | 33GB | 31GB | 31GB | 11GB |
AdaLoRA | 29GB | 24GB | 25GB | 8GB |
GRACE | 27GB | 23GB | 6GB |
Edit large language models(LLMs) around 5 seconds
Following example shows you how to perform editing with EasyEdit. More examples and tutorials can be found at examples
BaseEditor
is the class for Language Modality Knowledge Editing. You can choose the appropriate editing method based on your specific needs.
With the modularity and flexibility of EasyEdit
, you can easily use it to edit model.
Step1: Define a PLM as the object to be edited.
Choose the PLM to be edited. EasyEdit
supports partial models(T5
, GPTJ
, GPT-NEO
, LlaMA
so far) retrievable on HuggingFace. The corresponding configuration file directory is hparams/YUOR_METHOD/YOUR_MODEL.YAML
, such as hparams/MEND/gpt2-xl.yaml
, set the corresponding model_name
to select the object for knowledge editing.
model_name: gpt2-xl
model_class: GPT2LMHeadModel
tokenizer_class: GPT2Tokenizer
tokenizer_name: gpt2-xl
model_parallel: false # true for multi-GPU editing
Step2: Choose the appropriate Knowledge Editing Method The selection of editing methods is a crucial step, as different methods have their own strengths and weaknesses. Users need to consider the trade-off between editing success rate, generalization, and maintaining unrelated performance. For specific performance details of each method, please refer to the paper: Editing Large Language Models: Problems, Methods, and Opportunities.
## In this case, we use MEND method, so you should import `MENDHyperParams`
from easyeditor import MENDHyperParams
## Loading config from hparams/MEMIT/gpt2-xl.yaml
hparams = MENDHyperParams.from_hparams('./hparams/MEND/gpt2-xl')
Step3: Provide the edit descriptor and edit target
## edit descriptor: prompt that you want to edit
prompts = [
'What university did Watts Humphrey attend?',
'Which family does Ramalinaceae belong to',
'What role does Denny Herzig play in football?'
]
## You can set `ground_truth` to None !!!(or set to original output)
ground_truth = ['Illinois Institute of Technology', 'Lecanorales', 'defender']
## edit target: expected output
target_new = ['University of Michigan', 'Lamiinae', 'winger']
Step4: Combine them into a BaseEditor
EasyEdit
provides a simple and unified way to init Editor, like huggingface: from_hparams.
## Construct Language Model Editor
editor = BaseEditor.from_hparams(hparams)
Step5: Provide the data for evaluation Note that the data for portability and locality are both optional(set to None for basic editing success rate evaluation only). The data format for both is a dict, for each measurement dimension, you need to provide the corresponding prompt and its corresponding ground truth. Here is an example of the data:
locality_inputs = {
'neighborhood':{
'prompt': ['Joseph Fischhof, the', 'Larry Bird is a professional', 'In Forssa, they understand'],
'ground_truth': ['piano', 'basketball', 'Finnish']
},
'distracting': {
'prompt': ['Ray Charles, the violin Hauschka plays the instrument', 'Grant Hill is a professional soccer Magic Johnson is a professional', 'The law in Ikaalinen declares the language Swedish In Loviisa, the language spoken is'],
'ground_truth': ['piano', 'basketball', 'Finnish']
}
}
In the above example, we evaluate the performance of the editing methods about "neighborhood" and "distracting".
Step6: Edit and Evaluation
Done! We can conduct Edit and Evaluation for your model to be edited. The edit
function will return a series of metrics related to the editing process as well as the modified model weights.
metrics, edited_model, _ = editor.edit(
prompts=prompts,
ground_truth=ground_truth,
target_new=target_new,
locality_inputs=locality_inputs,
keep_original_weight=False
)
## metrics: edit success, rephrase success, locality e.g.
## edited_model: post-edit model
Step7: RollBack In sequential editing, if you are not satisfied with the outcome of one of your edits and you do not wish to lose your previous edits, you can use the rollback feature to undo your previous edit. Currently, we only support the GRACE method. All you need to do is a single line of code, using the edit_key to revert your edit.
editor.rolllback('edit_key')
In EasyEdit, we default to using target_new as the edit_key
We specify the return metrics as dict
format, including model prediction evaluations before and after editing. For each edit, it will include the following metrics:
rewrite_acc
$\rightarrow$ Reliablilty
rephrase_acc
$\rightarrow$ Generalization
locality
$\rightarrow$ Locality
portablility
$\rightarrow$ Portablility
{
"post": {
"rewrite_acc": ,
"rephrase_acc": ,
"locality": {
"YOUR_LOCALITY_KEY": ,
//...
},
"portablility": {
"YOUR_PORTABILITY_KEY": ,
//...
},
},
"pre": {
"rewrite_acc": ,
"rephrase_acc": ,
"portablility": {
"YOUR_PORTABILITY_KEY": ,
//...
},
}
}
prompts
and editing target_new
.rephrase_prompts
are required.prompts
and ground_truth
.
Note: the length needs to be equal to the edit prompts
MEND
SERAC
For above editing methods, pre-training of corresponding meta-networks or classifiers is required. Therefore, in EasyEdit, we provide a unified framework for pretraining the relevant network structures. Take the training MEND for example:
Step3: Provide the edit training set
The currently supported and available datasets are: zsre
and counterfact
(Google Drive). Please place them in the "data" directory and initialize the dataset_class (ZsreDataset
for zsre and CounterFactDataset
for counterfact) to load the corresponding training set.
train_ds = ZsreDataset('./data/zsre_mend_train.json', config=training_hparams)
eval_ds = ZsreDataset('./data/zsre_mend_eval.json', config=training_hparams)
Step4: Combine them into a Trainer
trainer = EditTrainer(
config=training_hparams,
train_set=train_ds,
val_set=eval_ds
)
Step5: Run and Edit Done! We can conduct Run and Evaluation.
trainer.run()
CHECKPOINT
will be saved to the path results_dir
.archive
field in the hparams file to CHECKPOINT
. EasyEdit will automatically load the corresponding pre-trained weights during the editing process(Go to edit).Training Example
from easyeditor import EditTrainer, MENDTrainingHparams, ZsreDataset
training_hparams = MENDTrainingHparams.from_hparams('hparams/TRAINING/MEND/llama-7b.yaml')
train_ds = ZsreDataset('./data/zsre/zsre_mend_train.json', config=training_hparams)
eval_ds = ZsreDataset('./data/zsre/zsre_mend_eval.json', config=training_hparams)
trainer = EditTrainer(
config=training_hparams,
train_set=train_ds,
val_set=eval_ds
)
trainer.run()
KnowEdit is a benchmark dataset of knowledge editing for LLMs. You can easily obtain KnowEdit from HuggingFace, HuggingFace, and ModelScope.
dataset | HuggingFace | HuggingFace | ModelScope |
---|---|---|---|
KnowEdit | [HuggingFace] | [WiseModel] | [ModelScope] |
We provide detailed scripts for user to easily use KnowEdit, please refer to examples.
We present editing results of the four metrics on LlaMA-2-7B using EasyEdit. We adopt ZsRE as the test dataset.
❗️❗️Editing
llama-2-7B
requires 40G+ VRAM on GPU. (OOM solution)
Reliability | Generalization | Locality | Portability | |
---|---|---|---|---|
FT | 56.94 | 52.02 | 96.32 | 0.07 |
SERAC | 99.49 | 99.13 | 100.00 | 0.13 |
IKE | 100.00 | 99.98 | 69.19 | 67.56 |
MEND | 94.24 | 90.27 | 97.04 | 0.14 |
KN | 28.95 | 28.43 | 65.43 | 0.07 |
ROME | 92.45 | 87.04 | 99.63 | 10.46 |
MEMIT | 92.94 | 85.97 | 99.49 | 6.03 |
We also present editing results of KnowEdit on LlaMA-2-7B using EasyEdit.
DataSet | Metric | SERAC | ICE | AdaLoRA | MEND | ROME | MEMIT | FT-L | FT |
---|---|---|---|---|---|---|---|---|---|
WikiData_recent | |||||||||
Edit Succ. | 98.68 | 60.74 | 65.61 | 76.88 | 85.08 | 85.32 | 71.18 | 31.24 | |
Portability | 63.52 | 36.93 | 47.22 | 50.11 | 37.45 | 37.94 | 48.71 | 15.91 | |
Locality | 100.00 | 33.34 | 55.78 | 92.87 | 66.2 | 64.78 | 63.7 | 3.65 | |
Fluency | 553.19 | 531.01 | 537.51 | 586.34 | 574.28 | 566.66 | 549.35 | 428.67 | |
ZsRE | |||||||||
Edit Succ. | 99.67 | 66.01 | 69.86 | 96.74 | 96.57 | 83.07 | 54.65 | 36.88 | |
Portability | 56.48 | 63.94 | 52.95 | 60.41 | 52.20 | 51.43 | 45.02 | 8.72 | |
Locality | 30.23 | 23.14 | 72.21 | 92.79 | 27.14 | 25.46 | 71.12 | 0.31 | |
Fluency | 410.89 | 541.14 | 532.82 | 524.33 | 570.47 | 559.72 | 474.18 | 471.29 | |
WikiBio | |||||||||
Edit Succ. | 99.69 | 95.53 | 97.02 | 93.66 | 95.05 | 94.29 | 66.27 | 95.64 | |
Locality | 69.79 | 47.90 | 57.87 | 69.51 | 46.96 | 51.56 | 60.14 | 13.38 | |
Fluency | 606.95 | 632.92 | 615.86 | 609.39 | 617.25 | 616.65 | 604.00 | 589.22 | |
WikiData_counterfact | |||||||||
Edit Succ. | 99.99 | 69.83 | 72.14 | 78.82 | 83.21 | 83.41 | 51.12 | 26.78 | |
Portability | 76.07 | 45.32 | 55.17 | 57.53 | 38.69 | 40.09 | 39.07 | 16.94 | |
Locality | 98.96 | 32.38 | 66.78 | 94.16 | 65.4 | 63.68 | 62.51 | 0.29 | |
Fluency | 549.91 | 547.22 | 553.85 | 588.94 | 578.84 | 568.58 | 544.80 | 483.71 | |
ConvSent | |||||||||
Edit Succ. | 62.75 | 52.78 | 44.89 | 50.76 | 45.79 | 44.75 | 49.50 | 61.93 | |
Locality | 0.26 | 49.73 | 0.18 | 3.42 | 0.00 | 0.00 | 0.00 | 0.00 | |
Fluency | 458.21 | 621.45 | 606.42 | 379.43 | 606.32 | 602.62 | 607.86 | 546.24 | |
Sanitation | |||||||||
Edit Succ. | 0.00 | 72.50 | 2.50 | 0.00 | 85.00 | 48.75 | 0.00 | 60.00 | |
Locality | 100.00 | 56.58 | 65.50 | 5.29 | 50.31 | 67.47 | 14.78 | 42.61 | |
Fluency | 416.29 | 794.15 | 330.44 | 407.18 | 465.12 | 466.10 | 439.10 | 351.39 |
❗️❗️ Please note that if you wish to reproduce the results regarding Rome on Knowedi, ensure that
fp16: False
.
locality
and portability
metrics.personality editing
, etc.Meanwhile, we will offer long-term maintenance to fix bugs, solve issues and meet new requests. So if you have any problems, please put issues to us.
Please cite our paper if you use EasyEdit in your work.
@article{zhang2024comprehensive,
title={A Comprehensive Study of Knowledge Editing for Large Language Models},
author={Zhang, Ningyu and Yao, Yunzhi and Tian, Bozhong and Wang, Peng and Deng, Shumin and Wang, Mengru and Xi, Zekun and Mao, Shengyu and Zhang, Jintian and Ni, Yuansheng and others},
journal={arXiv preprint arXiv:2401.01286},
year={2024}
}
@article{wang2023easyedit,
title={Easyedit: An easy-to-use knowledge editing framework for large language models},
author={Wang, Peng and Zhang, Ningyu and Xie, Xin and Yao, Yunzhi and Tian, Bozhong and Wang, Mengru and Xi, Zekun and Cheng, Siyuan and Liu, Kangwei and Zheng, Guozhou and others},
journal={arXiv preprint arXiv:2308.07269},
year={2023}
}
@article{yao2023editing,
title={Editing Large Language Models: Problems, Methods, and Opportunities},
author={Yao, Yunzhi and Wang, Peng and Tian, Bozhong and Cheng, Siyuan and Li, Zhoubo and Deng, Shumin and Chen, Huajun and Zhang, Ningyu},
journal={arXiv preprint arXiv:2305.13172},
year={2023}
}
@article{cheng2023edit,
title={Can We Edit Multimodal Large Language Models?},
author={Cheng, Siyuan and Tian, Bozhong and Liu, Qingbin and Chen, Xi and Wang, Yongheng and Chen, Huajun and Zhang, Ningyu},
journal={arXiv preprint arXiv:2310.08475},
year={2023}
}
@article{mao2023editing,
title={Editing personality for llms},
author={Mao, Shengyu and Zhang, Ningyu and Wang, Xiaohan and Wang, Mengru and Yao, Yunzhi and Jiang, Yong and Xie, Pengjun and Huang, Fei and Chen, Huajun},
journal={arXiv preprint arXiv:2310.02168},
year={2023}
}
@misc{knowlm,
author = {Ningyu Zhang and Jintian Zhang and Xiaohan Wang and Honghao Gui and Kangwei Liu and Yinuo Jiang and Xiang Chen and Shengyu Mao and Shuofei Qiao and Yuqi Zhu and Zhen Bi and Jing Chen and Xiaozhuan Liang and Yixin Ou and Runnan Fang and Zekun Xi and Xin Xu and Lei Li and Peng Wang and Mengru Wang and Yunzhi Yao and Bozhong Tian and Yin Fang and Guozhou Zheng and Huajun Chen},
title = {KnowLM Technical Report},
year = {2023},
url = {http://knowlm.zjukg.cn/},
}
We thank all the contributors to this project, more contributors are welcome!
🙌 We would like to express our heartfelt gratitude for the contribution of FastEdit, ROME, GRACE, MELO, PMET to our project, as we have utilized portions of their source code in our project. Many thanks to all the colleagues in the community for submitting issues and providing technical support.