DeveloperWeek 2025: Full Schedule

arrow_back View All Dates

9:30am PST

[Virtual] PRO Session: Striking the Balance: Leveraging Human Intelligence with LLMs for Cost-Effective Annotations

Thursday February 20, 2025 9:30am - 9:55am PST

Shambhavi Srivastava, Appen, AI Solutions Architect

Data annotation involves assigning relevant information to raw data to enhance machine learning (ML) model performance. While this process is crucial, it can be time-consuming and expensive. The emergence of Large Language Models (LLMs) offers a unique opportunity to automate data annotation. However, the complexity of data annotation, stemming from unclear task instructions and subjective human judgment on equivocal data points, presents challenges that are not immediately apparent.

In this session, Shambhavi Srivastava, AI Solutions Architect at Appen will provide a overview of an experiment that the company recently conducted to test the tradeoff between quality and cost of training ML models via LLMs vs human input. Their goal was to differentiate between utterances that could be confidently annotated by LLMs, and those that required human intervention. This differentiation was crucial to ensure a diverse range of opinions or to prevent incorrect responses from overly general models. Shambhavi will walk audience members through the dataset used as well as methodology for the experiment, as well as the company’s research findings.

Speakers

Shambhavi Srivastava

AI Solutions Architect, Appen

Thursday February 20, 2025 9:30am - 9:55am PST
VIRTUAL AI DevWorld Main Stage https://app.events.ringcentral.com/events/developerweek-productworld-ai-devworld-2025/reception

AI DevWorld

Tracks & Topics Gen AI / LLMs
Session Type PRO Session
Conferences AI DevWorld, Generative AI & LLMs (AI DevWorld), AI DevWorld: AI Strategy Conference, Applied AI Innovation (AI DevWorld), Virtual
In-Person/Virtual Virtual

11:30am PST

[Virtual] KEYNOTE (AI DevWorld): Block -- Goose: The AI-Driven Open Source Developer Agent

Thursday February 20, 2025 11:30am - 11:55am PST

VIRTUAL AI DevWorld Main Stage

Angie Jones, Block, Global Vice President of Developer Relations

Meet Goose, your new AI-driven open source sidekick for developers that doesn’t just chat—it takes action. Goose is an interactive agent that writes code, runs commands, and fixes bugs, making it more than just a code completion tool. Goose delivers huge value in real-world developer use cases, but we’re even more excited about its future as a general purpose agent that can dynamically operate new systems. In this session, we will share results from our internal alpha with the thousands of engineers at Block and plans for continued evolution.

Speakers

Angie Jones

Global Vice President of Developer Relations, Block

Angie Jones is the Global Vice President of Developer Relations at Block, Inc.An award-winning educator and international keynote speaker, Angie shares her extensive knowledge with software companies and conference audiences worldwide.As a Master Inventor, Angie is recognized for... Read More →

Thursday February 20, 2025 11:30am - 11:55am PST
VIRTUAL AI DevWorld Main Stage https://app.events.ringcentral.com/events/developerweek-productworld-ai-devworld-2025/reception

5. KEYNOTES & FEATURED

Tracks & Topics Open Source Strategy
Session Type OPEN Session
Conferences OPEN Session, AI DevWorld, AI DevWorld: AI/ML Engineering Conference, Applied Machine Learning (AI DevWorld), AI DevWorld: AI Strategy Conference, Applied AI Innovation (AI DevWorld), Virtual
In-Person/Virtual Virtual