WinkNLP – a journey to open source

Check Reference

– In this talk, Graype co-founder, Sanjaya, will share:

  1. our transition from working with closed source to open source; 
  2. the journey in developing winkNLP, our developer-friendly, JavaScript library for Natural Language Processing (NLP); 
  3. the challenges we have mitigated and continue to address as the library grows and becomes more robust.


Background: our modest start with small packages — mostly around NLP, and what encouraged us to take a plunge to develop a comprehensive NLP tool:

  1. User feedback and community encouragement
  2. Application of our packages by a range of projects like Social Analyzer and Trustroots, among others
  3. Community role in transitioning licence to MIT from GPL
  4. Our focus on standards, quality and documentation


Beginning: in a landscape dominated by NLTK, spaCy and coreNLP, we began developing an integrated NLP product in late 2018 with the following objectives:

  1. light weight
  2. developer friendly
  3. balance between performance and accuracy
  4. support for pure browser side and server side


Journey: Sanjaya will highlight how with a team of 2½ people we met our objectives and addressed the roadblocks we faced:

  1. the difficulty in finding labelled data with permissive licences for training models;
  2. and our alternate approach, where we:
  3.  created language models using hierarchical finite state machines
  4.  used limited open source data and complemented them with manual efforts
  5. developing a declarative syntax to make winkNLP developer friendly
  6. meeting performance and accuracy benchmarks
  7. outcome — quick demo on web browser
  8. learning from user feedback


– Finally, Sanjaya will talk about where we are today and headed next, including our:

  1. Current challenges: sustaining an open-source project, and keeping pace with a fast-changing technological landscape; and
  2. Future plans: to extend winkNLP to more languages that use the Latin script, as well as Indian languages; and our efforts to nurture a community around the project.  


Comments
Want to discuss?
Post it here, our mentors will help you out.