N., Hutchinson B., Pope R., Bradbury J., Austin J., Isard M., Gur-Ari G., Yin P., Duke T., Levskaya A., Ghemawat S., Dev S., Michalewski H., Garcia X., Misra V., Robinson K., Fedus L., Zhou D., Ippolito D., Luan D., Lim H., Zoph B., Spiridonov A., Sepassi R., Dohan D., Agrawal S., Omernick M., Dai A. M., Pillai T. S., Pellat M., Lewkowycz A., Moreira E., Child R., Polozov O., Lee K., Zhou Z., Wang X., Saeta B., Diaz M., Firat O., Catasta M., Wei J., Meier-Hellstern K., Eck D., Dean J., Petrov S., Fiedel N. (2022). PaLM: Scaling Language Modeling with Pathways // https://arxiv.org/abs/2204.02311
2636
Anil R., Dai A. M., Firat O., Johnson M., Lepikhin D., Passos A., Shakeri S., Taropa E., Bailey P., Chen Z., Chu E., Clark J. H., Shafey L. E., Huang Y., Meier-Hellstern K., Mishra G., Moreira E., Omernick M., Robinson K., Ruder S., Tay Y., Xiao K., Xu Y., Zhang Y., Abrego G. H., Ahn J., Austin J., Barham P., Botha J., Bradbury J., Brahma S., Brooks K., Catasta M., Cheng Y., Cherry C., Choquette-Choo C. A., Chowdhery A., Crepy C., Dave S., Dehghani M., Dev S., Devlin J., Díaz M., Du N., Dyer E., Feinberg V., Feng F., Fienber V., Freitag M., Garcia X., Gehrmann S., Gonzalez L., Gur-Ari G., Hand S., Hashemi H., Hou L., Howland J., Hu A., Hui J., Hurwitz J., Isard M., Ittycheriah A., Jagielski M., Jia W., Kenealy K., Krikun M., Kudugunta S., Lan C., Lee K., Lee B., Li E., Li M., Li W., Li Y., Li J., Lim H., Lin H., Liu Z., Liu F., Maggioni M., Mahendru A., Maynez J., Misra V., Moussalem M., Nado Z., Nham J., Ni E., Nystrom A., Parrish A., Pellat M., Polacek M., Polozov A., Pope R., Qiao S., Reif E., Richter B., Riley P., Ros A. C., Roy A., Saeta B., Samuel R., Shelby R., Slone A., Smilkov D., So D. R., Sohn D., Tokumine S., Valter D., Vasudevan V., Vodrahalli K., Wang X., Wang P., Wang Z., Wang T., Wieting J., Wu Y., Xu K., Xu Y., Xue L., Yin P., Yu J., Zhang Q., Zheng S., Zheng C., Zhou W., Zhou D., Petrov S., Wu Y. (2023). PaLM 2 Technical Report // https://arxiv.org/abs/2305.10403
2637
Chen X., Liang C., Huang D., Real E., Wang K., Liu Y., Pham H., Dong X., Luong T., Hsieh C.-J., Lu Y., Le Q. V. (2023). Symbolic Discovery of Optimization Algorithms // https://arxiv.org/abs/2302.06675
2638
Liu H., Li Z., Hall D., Liang P., Ma T. (2023). Sophia: A Scalable Stochastic Second-order Optimizer for Language Model Pre-training // https://arxiv.org/abs/2305.14342
2639
Tay Y., Dehghani M., Tran V. Q., Garcia X., Wei J., Wang X., Chung H. W., Shakeri s., Bahri D., Schuster T., Zheng H. S., Zhou D., Houlsby N., Metzler D. (2022). UL2: Unifying Language Learning Paradigms // https://arxiv.org/abs/2205.05131
2640
Змитрович Д. (2023). FRED-T5. Новая SOTA модель для русского языка от SberDevices. / Хабр, 19 апр 2023 // https://habr.com/ru/companies/sberdevices/articles/730088/
2641
Bavarian M., Jun H., Tezak N., Schulman J., McLeavey C., Tworek J., Chen M. (2022). Efficient Training of Language Models to Fill in the Middle // https://arxiv.org/abs/2207.14255
2642
Ouyang L., Wu J., Jiang X., Almeida D., Wainwright C. L., Mishkin P., Zhang C., Agarwal S., Slama K., Ray A., Schulman J., Hilton J., Kelton F., Miller L., Simens M., Askell A., Welinder P., Christiano P., Leike J., Lowe R. (2022). Training language models to follow instructions with human feedback // https://cdn.openai.com/papers/Training_language_models_to_follow_instructions_with_human_feedback.pdf
2643
Branwen G. (2022). GPT-3 2nd Anniversary / Reddit, May 28, 2022 // https://www.reddit.com/r/mlscaling/comments/uznkhw/gpt3_2nd_anniversary/
2644
OpenAI (2023). GPT-4 Technical Report // https://arxiv.org/abs/2303.08774
2645
Pichai S. (2023). An important next step on our AI journey // https://blog.google/technology/ai/bard-google-ai-search-updates/
2646
Anthropic PBC (2023). Introducing Claude // https://www.anthropic.com/index/introducing-claude
2647
SambaNova Systems, Together Computer (2023). BLOOMChat: a New Open Multilingual Chat LLM // https://huggingface.co/sambanovasystems/BLOOMChat-176B-v1
2648
Taori R., Gulrajani I., Zhang T, Dubois Y., Li X., Guestrin C., Liang P., Hashimoto T. B. (2023). Stanford Alpaca: An Instruction-following LLaMA model // https://github.com/tatsu-lab/stanford_alpaca
2649
Touvron H., Lavril T., Izacard G., Martinet X., Lachaux M.-A., Lacroix T., Rozière B., Goyal N., Hambro E., Azhar F., Rodriguez A., Joulin A., Grave E., Lample G. (2023). LLaMA: Open and Efficient Foundation Language Models // https://arxiv.org/abs/2302.13971
2650
Zhang S., Roller S., Goyal N., Artetxe M., Chen M., Chen S., Dewan C., Diab M., Li X., Lin X. V., Mihaylov T., Ott M., Shleifer S., Shuster K., Simig D., Koura P. S., Sridhar A., Wang T., Zettlemoyer L. (2022). OPT: Open Pre-trained Transformer Language Models // https://arxiv.org/abs/2205.01068
2651
Taori R., Gulrajani I., Zhang T, Dubois Y., Li X., Guestrin C., Liang P., Hashimoto T. B. (2023). Stanford Alpaca: An Instruction-following LLaMA model // https://github.com/tatsu-lab/stanford_alpaca
2652
Vicuna Team (2023). Vicuna: An Open-Source Chatbot Impressing GPT-4 with 90%* ChatGPT Quality // https://lmsys.org/blog/2023-03-30-vicuna/
2653
Dettmers T., Pagnoni A., Holtzman A., Zettlemoyer L. (2023). QLoRA: Efficient Finetuning of Quantized LLMs // https://arxiv.org/abs/2305.14314
2654
Geng X., Gudibande A., Liu H., Wallace E., Abbeel P., Levine S., Song D. (2023). Koala: A Dialogue Model for Academic Research // https://bair.berkeley.edu/blog/2023/04/03/koala/
2655
Patil S. G., Zhang T., Wang X., Gonzalez J. E. (2023). Gorilla: Large Language Model Connected with Massive APIs // https://arxiv.org/abs/2305.15334
2656
Mukherjee S., Mitra A., Jawahar G., Agarwal s., Palangi H., Awadallah A. (2023). Orca: Progressive
