Keyfiyyət laboratoriyası
AI-ı necə test edirik: son qiymətləndirmənin real rəqəmləri, uğursuzluqlar daxil.
Son icra: 9 okt 2026, 14:23 · çıxarıcı: heuristic · claude-haiku-5-5 (offline) / claude-sonnet-5-5
Bu icra oflayn evristik çıxarıcıdan istifadə etdi (API açarı yoxdur). LLM-i ölçmək üçün API açarı ilə icra edin.
Qızıl etiketlər profillərlə birlikdə yaradılıb və insan yoxlamasını gözləyir (48-dən 0 yoxlanılıb).
Çıxarma dəqiqliyi
84%
n = 48
Precision@3
73%
baza 22%
Ədalətlilik testləri
24/24
τ min = 1.00
Dayanıqlıq testləri
17/17
Əsaslandırılmış izahlar
45/45
+0 risks by validator
Orta çıxarma müddəti
17 ms
p95 85 ms
Bir profilin xərci
$0.00041
estimated
100 nəfərlik tədbirin xərci
$0.0410
estimated
İlk 3-də pis uyğunluq
0%
baseline 33%
nDCG@5
0.76
baseline 0.28
Sahələr və dillər üzrə çıxarma dəqiqliyi
Sıralama keyfiyyəti: TeamMatch və açar söz bazası
15 scenarios · end-to-end P@3 with extracted profiles: 73%
Hakaton bölgüsü və təsadüfi
Eyni insanların 200 təsadüfi bölgüsünün ortalaması. · 148 ms-də hesablandı
Ədalətlilik testləri
24 kontrfaktual cütdən 24-i eyni sıralama verir (Kendall τ = 1).
Structured profiles identical on scoring fields: 24/24
Rankings identical (Kendall τ = 1), seeker and candidate direction
Explanations free of protected terms (az/ru/en regex): 24/24
Dayanıqlıq testləri
- typos
- slang
- transliteration
- code-switching
- very short
- long irrelevant
- emojis
- prompt injection (en)
- prompt injection (az)
- prompt injection (ru)
- self-contradiction
- missing hours
- negated skill
- three languages
- evening hours
- protected info only + role
- shouting
Uğursuz hallar
Yalnız uğurları yox, real uğursuzluqları da göstəririk.
| Kateqoriya | Giriş | Gözlənilən | Alınan | Ehtimal olunan səbəb | Növbəti addım |
|---|---|---|---|---|---|
| extraction s25 | Аналитик данных: SQL, Python, Power BI. 3 года в ритейле. Хочу участвовать в хакатонах и прокачать ML. По вечерам, часов… | secondaryRoles=[]; hours=[8,12]; commitment="part_time"; goals=["win_hackathon","learn"] | secondaryRoles=["backend"]; hours=null; commitment="unknown"; goals=["win_hackathon"] | Time stated indirectly (evenings, weekends, a range, or 'full-time but…'). | Normalize time/role phrases before extraction; review the gold label. |
| extraction s47 | Менеджер по работе с клиентами в медицинском центре, 3 года. Умею продавать услуги клиникам и врачам. Хочу в healthtech-… | primaryRole="sales_bd"; secondaryRoles=["domain_expert"]; skills=["b2b sales","negotiation"] | primaryRole="domain_expert"; secondaryRoles=[]; skills=[] | Role signal is ambiguous or a negated/sought role was read as the person's own role. | Add a few-shot example for this phrasing; extend taxonomy aliases. |
| extraction s04 | Full-stack dev (React + Node.js), 2 years freelancing, built 6 client websites. I want to build my own product and I'm r… | secondaryRoles=[]; goals=["build_startup"]; lookingForRoles=["product","sales_bd","design_ux"] | secondaryRoles=["frontend"]; goals=["join_startup"]; lookingForRoles=["design_ux"] | Sought roles were implied ('technical co-founder', 'need developers') rather than named. | Normalize time/role phrases before extraction; review the gold label. |
| extraction s03 | Привет! Я бэкенд-разработчик, Java/Spring, 5 лет опыта в банке. Работаю full-time, на стартап могу выделять вечера, часо… | skills=["java","sql"]; hours=[8,12]; commitment="part_time" | skills=["java"]; hours=40; commitment="full_time" | Time stated indirectly (evenings, weekends, a range, or 'full-time but…'). | Normalize time/role phrases before extraction; review the gold label. |
| extraction s37 | бекенд разраб, пхп ларавел 2 года, mysql. ищу стартап куда можно влится. могу по вечерам | skills=["php","sql"]; hours=[8,15]; commitment="part_time" | skills=["sql"]; hours=null; commitment="unknown" | Time stated indirectly (evenings, weekends, a range, or 'full-time but…'). | Normalize time/role phrases before extraction; review the gold label. |
| extraction s42 | Marketoloqam. SMM və target reklam. Tam ştat işləyə bilərəm, amma yalnız həftəsonları. Startapa qoşulmaq istəyirəm. | skills=["smm","digital marketing"]; hours=[8,16]; commitment="weekends_only" | skills=["smm"]; hours=40; commitment="full_time" | Time stated indirectly (evenings, weekends, a range, or 'full-time but…'). | Normalize time/role phrases before extraction; review the gold label. |
| ranking s06 | s06: sales student (weekends, az) needs a technical co-founder | top-3 ⊂ {s27, s10, s01, s29} | top-3 = s27(backend), s10(backend), s45(backend) | s45 (backend, part_time) fill the needed role but the human label preferred others (time/goal judgement differs from the weights). | Review the gold label; if it holds, tune time/goal weights or add a hard filter. |
| ranking s14 | s14: ML founder (AI chatbot, az, 30h) needs sales/product | top-3 ⊂ {s28, s33, s11, s05, s38} | top-3 = s33(sales_bd), s38(product), s29(fullstack) | s29 (fullstack, part_time) fill the needed role but the human label preferred others (time/goal judgement differs from the weights). | Review the gold label; if it holds, tune time/goal weights or add a hard filter. |
| ranking s21 | s21: hardware founder (agritech IoT) needs backend/mobile/sales | top-3 ⊂ {s10, s27, s28, s01, s33, s05} | top-3 = s33(sales_bd), s28(sales_bd), s29(fullstack) | s29 (fullstack, part_time) fill the needed role but the human label preferred others (time/goal judgement differs from the weights). | Review the gold label; if it holds, tune time/goal weights or add a hard filter. |
| ranking s26 | s26: doctor with clinic idea (7h, az) needs backend | top-3 ⊂ {s27, s10, s01, s45} | top-3 = s29(fullstack), s17(backend), s06(sales_bd) | s29 (fullstack, part_time), s17 (backend, part_time), s06 (sales_bd, weekends_only) fill the needed role but the human label preferred others (time/goal judgement differs from the weights). | Review the gold label; if it holds, tune time/goal weights or add a hard filter. |
| ranking s10 | s10: backend founder (fintech, az/ru) needs sales | top-3 ⊂ {s05, s28, s33, s15} | top-3 = s28(sales_bd), s07(marketing_growth), s33(sales_bd) | s07 (marketing_growth, part_time) fill the needed role but the human label preferred others (time/goal judgement differs from the weights). | Review the gold label; if it holds, tune time/goal weights or add a hard filter. |
| ranking s43 | s43: full-stack ex-CTO (full-time) needs sales/product | top-3 ⊂ {s47, s11, s28, s24} | top-3 = s15(sales_bd), s47(sales_bd), s11(product) | s15 (sales_bd, full_time) fill the needed role but the human label preferred others (time/goal judgement differs from the weights). | Review the gold label; if it holds, tune time/goal weights or add a hard filter. |
| ranking s11 | s11: product manager (full-time, az) needs engineers | top-3 ⊂ {s39, s29, s04, s23} | top-3 = s04(fullstack), s43(fullstack), s29(fullstack) | s43 (fullstack, full_time) fill the needed role but the human label preferred others (time/goal judgement differs from the weights). | Review the gold label; if it holds, tune time/goal weights or add a hard filter. |
| ranking s22 | s22: product manager (edtech, Russian only) needs developers | top-3 ⊂ {s37, s03, s31, s10} | top-3 = s27(backend), s03(backend), s37(backend) | s27 (backend, part_time) fill the needed role but the human label preferred others (time/goal judgement differs from the weights). | Review the gold label; if it holds, tune time/goal weights or add a hard filter. |
| ranking s44 | s44: agronomist (Russian only) needs a technical person | top-3 ⊂ {s37, s03, s10, s27} | top-3 = s03(backend), s37(backend), s31(backend) | s31 (backend, part_time) fill the needed role but the human label preferred others (time/goal judgement differs from the weights). | Review the gold label; if it holds, tune time/goal weights or add a hard filter. |
| ranking s09 | s09: junior frontend (portfolio) needs design + backend | top-3 ⊂ {s02, s45, s17, s13} | top-3 = s02(design_ux), s45(backend), s16(marketing_growth) | s16 (marketing_growth, part_time) fill the needed role but the human label preferred others (time/goal judgement differs from the weights). | Review the gold label; if it holds, tune time/goal weights or add a hard filter. |
Bu gün və TeamMatch
| Bu gün (Telegram postu / təsadüfi komandalar / açar söz axtarışı) | TeamMatch | |
|---|---|---|
| Tələb olunan rolu çatışmayan komandalar | 91% | 67% |
| İlk 3-də uyğun namizəd (P@3) | 22% | 73% |
| İlk görüşdən əvvəl görünən risklər | 0 | hər uyğunluqda orta hesabla 1.2 |
| Qısa siyahıya qədər vaxt | istifadəçi müsahibələrindən doldurulacaq | < 9 s |
Xərclər qeydə alınmış token istifadəsi × dərc olunmuş model qiymətləri əsasında. Estimated: no live calls in this run. Extraction ≈ 1847+450 tokens on claude-haiku-5-5; explanation ≈ 1282+450 on claude-sonnet-5-5. · Bir profilin xərci: $0.00041 · user flow: $0.0358