Retry (F1) va circuit breaker (qoida 36)

qa-ep-13-t03-s03 statusni o'zgartirdi Review → Bajarildi

qa-ep-13-t03-s03 izoh qoldirdi

QA (qa-EP-13-T03-S03) — TEKSHIRILDI, `done`. Commit a705bb5 + QA test commit'i 2a39f90. O'LCHANGAN DALIL — to'plam 3 marta ketma-ket yugurtirildi (beqarorlik yo'q): `mvn -o test -Dtest='Notification*Test,PushTokens*Test,AdminSecurityNotification*Test,Fcm*Test'` run1: Tests run 131 / Failures 0 / Errors 0 / 15 klass (poydevor 114/12 dan kam emas) run2, run3 (QA testi qo'shilgach): Tests run 132 / Failures 0 / Errors 0 / 15 klass. Yangi klasslar yurgani tasdiqlandi: NotificationCircuitBreakerTest (8), NotificationOutboxWorkerCircuitBreakerTest (3), NotificationRetryBackoffJitterTest (5). MEZONLAR (har biri kod darajasida, alohida): - 5567 ✓ NotificationOutboxWorkerTest#allFiveF1IntervalsAreCheckedIndividuallyWithinJitterBoundsThenRowDies: n=1..4 ORALIQLARI ALOHIDA, ham chegara (0.85..1.15 x nominal), ham nazorat qilingan jitter bilan aniq qiymat; 5-chi urinish -> `dead`, attempt_count=5. n=5 (16 s) formula darajasida NotificationRetryBackoffJitterTest da. Jitter tasodifga bog'liq EMAS — NotificationJitterSource seam (FixedNotificationJitterSource), 3 yurishda barqaror. - 5568 ✓ NotificationOutboxWorkerCircuitBreakerTest:104 — 25 qator, FAQAT 20 tasi FCM'ga urinadi, breaker OPEN, 25 tasi ham `pending`, attempt_count boshlang'ich qiymatida (fixture 0/2/4 — turli), keyingi tick claim=0. - 5569 ✓ ...:218 — 60 s dan keyin half-open batch=1 sinovi o'tadi, breaker to'liq yopiladi, 6 qator ham `sent`, outage davomida hech biri `dead` emas. - 5570 ✓ ...:157 — UNREGISTERED dan keyin AYNAN 19 tizimli xato yetarli EMAS (18 bo'lsa edi — hisoblangan bo'lardi); token xatosi -> `dead`, tizimli xato -> attempt_count tegilmagan. MUTATSIYA (izolyatsiyalangan `git worktree` da, har mutant javap bilan bytecode'da tasdiqlangan) — 10 mutant: 1) breaker chegarasi 20->1: O'LDI (5 fail) 2) qoida 36 b requeue olib tashlandi: O'LDI (5568+5569) 3) tizimli xatoda attempt_count++: O'LDI (3/3) 4) token xatosi tizimli deb sanaldi: O'LDI (5570, :191) 5) F1 2^(n-1) -> chiziqli: O'LDI (6 fail) 6) jitter +-0.15 -> +-0.60: O'LDI (4 fail) 7) attempt_count=5 da `failed` qoldirish: O'LDI (2 fail) 8) half-open ikkilanishi olib tashlandi: O'LDI 9) 🔴 `AND status = 'sending'` beshta finalize UPDATE'dan olib tashlandi: TIRIK QOLDI — 131 test YASHIL qoldi (javap: bytecode'da qo'riqchi 0, `IS NOT NULL` 5 marta). Ya'ni S01 QA topilmasining TUZATISHI kodda bor, lekin testda qamralmagan edi. Asosiy kodga TEGILMADI — QA test qo'shdi: NotificationOutboxWorkerTest#finalizeUpdatesRefuseToChangeRowsThatAreNoLongerSending (commit 2a39f90): `sent`/`dead`/`pending` qatorga 5 finalize metodning hammasi urinib ko'riladi -> qator o'zgarmaydi; teskari tomon — haqiqiy `sending` finalize qilinadi. Qayta o'lchov: mutant 9 endi O'LADI (:369), faqat markSent'dan olib tashlangan 9b varianti ham O'LADI (:386, aynan `dead -> sent`). `AND status='sending'` QO'RIQCHISI TO'LIQ: markSent(166) / markFailedRetry(182) / markDeadImmediate(201) / markDeadAfterAttempt(218) / requeueAfterSystemicFailure(248) — beshtasi ham qo'riqlangan, qolib ketgani YO'Q; 0 qator holatida warnIfLostUpdate (NotificationOutboxRepository:262) log yozadi (qoida 29), jimgina yo'qolmaydi. BOSHQA TEKSHIRUVLAR: `openapi/` va `db/migration/` diff'i BO'SH (2a045d3..a705bb5) — kontrakt/migratsiya tegilmagan; breaker holati in-process (AR-5), DB ustuni yo'q; Thread.sleep / Awaitility YO'Q (butun vaqt mantiqi Clock orqali); testlar sanadan mustaqil (created_at = DB now(), o'tgan oyga tushiradigan INSERT yo'q); konfiguratsiya app.notifications.breaker.{failure-threshold,open-duration,max-open-duration} = 20/60s/5m DEFAULT (application.yml 285–297, env bilan almashtiriladi); log'da token/parol YO'Q; regressiya joyida: oneOfThreeTokensInvalidated... (sent_device_count=2), FcmConfigTest hasSingleBean(ApplicationRunner) , NotificationWorkerProdProfileTest 5 test, application-prod.yml `${NOTIFICATION_OUTBOX_WORKER_ENABLED:true}` va application-dev.yml `false` saqlangan; SimpleNotificationRetryBackoff o'chirilgan, unga havola faqat javadoc izohida. Fixture xilma-xilligi: attempt_count 0/2/4/5, xato kodlari 4 tizimli + UNREGISTERED + QUOTA_EXCEEDED, severity info/security, token soni 1 va 3, aralash holat (token xatosi + tizimli xato ketma-ketligi) — qoplangan. 📌 PM UCHUN IKKI ESLATMA (bu bargni bloklamaydi, tuzatish TALAB QILINMADI): (1) SPEC F1 arifmetikasi ichida ziddiyat: §6 jadvalida n=5 uchun "Hodisadan beri ~31 s" (1+2+4+8+16 yig'indisi), lekin §4 holat mashinasi `failed (attempt_count = 5) -> dead` deydi — demak amalda FAQAT 4 kutish (1+2+4+8 = 15 s) materiallashadi, 16 s hech qachon ishlatilmaydi. Kod holat mashinasiga amal qildi (to'g'ri tanlov, chunki §4 normativ va bir ma'noli). Spec matnini aniqlashtirish — 10_notifications §6 F1 izohi (§5 sikli orqali). (2) MAX_ATTEMPTS=5, BATCH_SIZE=100, LEASE_DURATION=5daq NotificationOutboxWorker'da hardcoded — S01 dan kelgan meros (S03 kiritmagan), S03 ning o'z raqamlari (20/60s/5daq) konfiguratsiyada. (3) Board'dagi `files` ro'yxatida yolg'on yozuv bor: `"x"` (kosmetik, kodda bunday fayl yo'q).

qa-ep-13-t03-s03 qabul mezonini belgiladi

✓ Bitta token xatosi (`UNREGISTERED`) breaker hisoblagichini oshirmaydi

qa-ep-13-t03-s03 qabul mezonini belgiladi

✓ FCM tiklangach kutish oynasidan keyin hammasi yetkaziladi va hech biri `dead` bo'lib qolmaydi

qa-ep-13-t03-s03 qabul mezonini belgiladi

✓ 20 ta ketma-ket 5xx dan keyin worker to'xtaydi, qatorlar `pending` da qoladi va `attempt_count` oshmaydi (AC15)

qa-ep-13-t03-s03 qabul mezonini belgiladi

✓ 5 urinishdan keyin qator `dead` bo'ladi va `next_attempt_at` oraliqlari F1 jadvaliga (jitter chegarasida) mos

coder-ep-13-t03-s03 statusni o'zgartirdi Ishlanyapti → Review

coder-ep-13-t03-s03 izoh qoldirdi

F1 jitter (JitteredNotificationRetryBackoff + NotificationJitterSource seam) va circuit breaker (NotificationCircuitBreaker, in-process, AR-5) qo'shildi. Worker: tizimli xato (UNAVAILABLE/INTERNAL/http_5xx/network_error) endi attempt_count'ga tegmaydi, requeueAfterSystemicFailure orqali to'g'ridan-to'g'ri pending'ga qaytadi; QUOTA_EXCEEDED va token xatolari (qoida 16) breaker hisoblagichiga tegmaydi (mezon 5570). Breaker OPEN bo'lganda claim to'xtaydi, lease'dagi qolgan qatorlar FCM'siz flush qilinadi (mezon 5568); half-open bitta sinov, muvaffaqiyatsizlikda kutish 60s->120s->240s->max 5 daq ikkilanadi (mezon 5569). Repository: barcha finalize UPDATE'larga AND status='sending' qo'riqchisi qo'shildi (S01 QA topilmasi, majburiy uzatish) + 0-qator holatida log.warn kuzatuv signali; terminal-holat lost-update senariysi test bilan qo'riqlangan (NotificationOutboxWorkerTest.sentAndDeadRowsAreNeverTouchedAgain allaqachon bor edi, endi repository qatlamida ham qo'riqchi ishlaydi). Konfiguratsiya: app.notifications.breaker.{failure-threshold,open-duration,max-open-duration} (default 20/60s/5m). Test: rm -rf target/surefire-reports + mvn -o test -Dtest='Notification*Test,PushTokens*Test,AdminSecurityNotification*Test,Fcm*Test' -> Tests run: 131, Failures: 0, Errors: 0 (15 klass; poydevor 114/12 dan ortiq). Yangi klasslar filtrga sig'ishi uchun JitteredNotificationRetryBackoffTest -> NotificationRetryBackoffJitterTest deb nomlandi. git grep MUTANT HEAD -- src/ bo'sh.

coder-ep-13-t03-s03 yangiladi

Yangilandi: branch, commit_sha

coder-ep-13-t03-s03 qabul mezonini belgiladi

✓ Bitta token xatosi (`UNREGISTERED`) breaker hisoblagichini oshirmaydi

coder-ep-13-t03-s03 qabul mezonini belgiladi

✓ FCM tiklangach kutish oynasidan keyin hammasi yetkaziladi va hech biri `dead` bo'lib qolmaydi

coder-ep-13-t03-s03 qabul mezonini belgiladi

✓ 20 ta ketma-ket 5xx dan keyin worker to'xtaydi, qatorlar `pending` da qoladi va `attempt_count` oshmaydi (AC15)

coder-ep-13-t03-s03 qabul mezonini belgiladi

✓ 5 urinishdan keyin qator `dead` bo'ladi va `next_attempt_at` oraliqlari F1 jadvaliga (jitter chegarasida) mos

coder-ep-13-t03-s03 fayllarni qayd etdi

16 ta fayl qayd etildi

coder-ep-13-t03-s03 fayllarni qayd etdi

1 ta fayl qayd etildi

coder-ep-13-t03-s03 statusni o'zgartirdi Backlog → Ishlanyapti

Agent ishni boshladi

coder-ep-13-t03-s03 ishga oldi

worker-ep-13-t03 izoh qoldirdi

UZATISH S01-QA -> S03 (majburiy; kod izohiga tayanmaslik uchun board'ga yozildi). S01 QA non-blocking defekt topdi, u AYNAN S03 tegadigan kod yo'lida: NotificationOutboxRepository dagi finalize UPDATE'larida (markSent / markDead* / markFailedRetry) `AND status = 'sending'` qo'riqchisi YO'Q - kafolat faqat claim SQL'ida. Bitta instansda xavfsiz (ADR-001 AR-5), LEKIN ikkinchi instans qo'shilsa yoki bitta qatorni ishlash lease muddatidan (5 daqiqa) oshsa: qator qayta claim qilinadi va eski ijro finalize qilib `dead -> sent` yoki `sent -> failed` yozib qo'yishi mumkin. Bu spec §4 da ATAYLAB TAQIQLANGAN o'tish (`sent -> *` va `dead -> *` terminal). S03 ishi: retry/finalize yo'lini o'zgartirayotganda har finalize UPDATE'iga `AND status = 'sending'` qo'riqchisini qo'sh (yangi scope emas - spec §4 ga muvofiqlik) va 0 qator qaytganda holatni o'zgartirmasdan (jimgina yo'qotmasdan) log/kuzatuv signali ber - qoida 29. Buni test bilan qo'riqla: terminal holatdagi qatorni finalize qilishga urinish holatni O'ZGARTIRMASIN.

pm-agent yaratdi

Retry (F1) va circuit breaker (qoida 36)