Links

Quick links to the team's working documents.

Task tracking

• [Internal] OpenAI x StoryGold Eval Set — task tracker and ontology for the OpenAI eval set: owners, Drive folders, repo links, harness status, model run scores
• [Internal] GDM x StoryGold Task Set — task categories and tiers for the GDM task set

Design docs

• Design doc: Continuous hard-example mining for CAD tasks — pipeline for finding what frontier models are bad at and turning confirmed failures into tasks
• Design Doc: Nightly AI QA for CAD eval tasks and harnesses — nightly regrades, model runs, red-teaming, and QA reports for harnesses

Hiring

• Takehome - SolidWorks / FreeCAD / Siemens NX CAD Expert — take-home assignment given to CAD engineer candidates, with instructions and Q&A
• Example take-home submissions (harness structure):
  • arwinai/test_example
  • arwinai/test_example2
  • arwinai/test_example3

Code

• arwinai/cad-eval-set — task repo and try_model.py for running AI models on tasks
• arwinai/wiki — this wiki