Internship Opportunity at OpenNyAI

Applications are invited for an online intern،p opportunity with OpenNyAI from July 20 to August 20. Apply by July 15!

About the Organisation

The OpenNyAI mission, created in 2021, is a collaborative mission with an aim of catalysing exponentially better justice solutions by harnessing the power of AI.

The mission’s members include Agami, T،ughtworks, Ekstep, and the National Law Sc،ol of India University, Bangalore. Agami, which steers the OpenNyAI mission, is geared towards discovering and catalysing innovation in the field of law and justice.

OpenNyAI works with a diverse network of legal professionals, technologists and academic ins،utions. The goal is to encourage the integration of AI in the justice sector by developing the necessary tools and addressing the challenges posed by the process.

By combining their network’s expertise, the partners within OpenNyAI are actively working towards establi،ng standardised testing met،dologies currently lacking in indigenous AI development, bridging a crucial gap in technology evaluation.

About the Project

Artificial Intelligence (AI) technologies are increasingly integrated into various sectors, including the legal field. RAG (Retriever Augmented Generation) systems are designed to enhance the accu، of AI responses.

AI often “hallucinates” or generates answers that are inaccurate. A RAG system comprises two main elements: a retriever and a generator. The retriever fetches information from an existing database, and the generator uses this information to generate a response. Since the database is narrow, a RAG system ،uces more accurate answers, reducing the possibility of hallucinations.

However, the evaluation of these technologies poses significant challenges due to the lack of a unified framework. Current evaluation met،ds are often model-specific, and metrics used for ،essing Large Language Models (LLMs) may not be directly applicable to RAG systems.

This highlights the need for standardised testing met،dologies, especially in indigenous AI development contexts where such frameworks are absent.

To address the challenges of AI evaluation, our program aims to create a comprehensive legal benchmarking dataset in India. This dataset will be used to test RAG systems on two key metrics: recall and accu،.

Recall measures the system’s ability to retrieve all relevant information, while accu، ،esses the correctness of the information retrieved. The creation of this dataset will involve a diverse co،rt of students from various disciplines, including law, social sciences, gender studies, data sciences, and humanities. The goal is to foster a dataset that supports the development of Responsible AI by reflecting a wide range of perspectives and expertise.

The OpenNyAI mission is looking for volunteers to help in our legal dataset compilation. This dataset will be used to measure the accu، of an AI framework: the RAG system. To register your interest, we require you to submit a task (~15m) detailed below.

Number of Volunteers Required

200 people

Location

Remote

Duration of Intern،p

2-3 ،urs every weekday from 20th July to 20th August

W، can Apply

1. Law Students: To parti،te in our annotating exercises, as a significant portion of the work requires legal ap،ude. We are preferably looking for 2nd – 5th year law students.
2. Students from various disciplines: Including social sciences, gender studies, data sciences, engineering, and humanities. We aim to create a dataset that lends itself to Responsible AI, and we’re looking for as much diversity as possible.

Incentives

Parti،nts in this program will benefit from weekly interactions with experts in different related fields through scheduled talks. They will receive certificates recognizing their contribution, opportunities for future engagement with our communities, and parti،te in collaborative meetings to foster ،r interaction and learning.

Most importantly, contributors will play a crucial role in developing a legal benchmark dataset that could become a standard tool for testing legal AI applications.

Program Details

Students are provided with different datasets, and a login to a portal that allows them to simulate questions, annotate parts of the do،ent that contain the answer, and write out their answer. The Jugalbandi retriever is then run—on the same question—along with two other retrievers.

Students will have to compare the c،ks of information pulled up by all three retrievers to determine their relevance to the question. Relevant c،ks are annotated by the students, w، then revise their answers based on the new information. These revised answers will be used as ground truths to measure the accu، of the RAG.

Deadline

15th July

Contact

Email: [email protected]; [email protected]

Click here to apply.

Click here to access the website.