랩장 추천 논문 & Technical Report 리스트

최신 이슈 및 Litmap 을 통한 주요 키논문 파악

• OpenAI (2024) SORA: Video generation models as world simulators https://openai.com/sora#research
• Google (2024) Our next-generation model: Gemini 1.5, https://blog.google/technology/ai/google-gemini-next-generation-model-february-2024/#gemini-15
  • Gemini 1.5: Our next-generation model, now available for Private Preview in Google AI Studio https://developers.googleblog.com/2024/02/gemini-15-available-for-private-preview-in-google-ai-studio.html

LLM 논문 리스트

그 외 읽어볼 논문 목록

Upstage(2024) 필수 논문 리스트 (랩장의 의견: GPT-1 부터 LLM 처음 시작하시는 분들을 위한 키논문 리스트)

Large-Scale Raw Corpus

• Common
• Pile - "The Pile: An 800GB Dataset of Diverse Text for Language Modeling
• Common Crawal
• The Pile- "An 800GB Dataset of Diverse Text for Language Modeling, Gao et al., arXiv, 2020"

Data Processing

1. Quality Filtering
2. Duplication

• Deduplicating Training Data Mitigates Privacy Risks in Language Models, Kandpal et al., ICML, 2022
• Deduplicating Training Data Makes Language Models Better, Lee et al., ACL, 2022