의료질문에 답변하는 llama 기반의 모델이 chatdoctor에 이어서 medalpaca라는 이름으로 출시되었습니다.

chatdoctor보다 dataset이 더 좋아졌습니다.

We will persistently update and refine the dataset, and we welcome everyone to contribute more 'grass' to Medical Meadow!

Source	n items
ChatDoc large	200000
Wikidoc	67704
Stackexchange academia	40865
Anki flashcards	33955
Stackexchange biology	27887
Stackexchange fitness	9833
Stackexchange health	7721
Wikidoc patient information	5942
Stackexchange bioinformatics	5407

다음은 벤치마크 결과입니다. 

We are benchmarking all models on the USMLE self assessment, which is available at this link. Note, that we removed all questions with images, as our models are not multimodal.

Model	Step1	Step2	Step3
LLaMA 7b	0.174	0.109	nan
Alpaca 7b naive	0.243	0.222	0.329
Alpaca 7b LoRA	0.261	0.264	0.266
ChatDoctor	0.187	0.185	0.148
MedAlpaca 7b	0.261	0.300	0.363
MedAlpaca 7b LoRA 8bit	0.196	0.209	0.185
MedAlpaca 13b LoRA 8bit	0.217	0.155	0.234
MedAlpaca 30b LoRA 8bit	0.315	0.327	0.355

*ChatDoctor 점수가 생각보다 높지 않네요. 

https://huggingface.co/medalpaca

kbressem/medAlpaca: LLM finetuned for medical question answering (github.com)