My involvement with OA

1. I started off as a contributor in the early days of the project (Jan 2023)
2. Worked in dataset creation, finetuning, and model alignment
3. Responsible for building some of the finest open-source chatgpt3.5 alternatives.
4. One and only model code owner from India.

Talking points

1. Brief description of the project
2. Early days - vision, roadmap, and initial challenges
3. Distribution, community, and compute.
4. Breakup of different processes involved in creating a high-quality chat model
5. dataset creation
6. finetuning
7. alignment.
8. Challenges faced in each of these steps
9. Scaling model training 3b to 70b parameters
10. Dataset collection procedure behind the OpenAssistant Conversations dataset and its importance to OSS AI. Paper https://arxiv.org/abs/2304.07327
11. Demo of one of finest OA models.
12. Future of OSS AI and challenges to be solved.
13. The Compute poor
14. Dense to Sparse models
15. lack of feedback data for model alignment