The three things that made the deferral inert are all fixed, in the order they were
found: the reply ring could not hold an answer across the ~1,000 records between two
creates; a residency budget could not help because what displaces an answer is what
the SERVER sends, not what the client emits; and so the mechanism had to be retention.
WHAT LANDED
* ILink::DeclareReplyRead + LinkCapabilities::RetainsReplies. The wire declares a
seq will be read BEFORE it publishes, so the declaration cannot race its own
answer; a link with the capability false ignores it, and the window does not
defer on one (CreateWindowEffective() is gated on it).
* StreamLink's ring of the last 8 answers becomes a declared-answer store: a
monotone whitelist plus a cursor, answers kept until their reader TAKES them
(take-and-delete under one lock, bytes swapped into the scratch), and a capacity
of 24 that is FATAL to exceed rather than evicting. An evicted wanted answer is
unrecoverable - re-latching by sending another create wipes the server's record
(PipeApply.cpp:2033) and its PendingUploads (PipeApply.h:332) while the client
cleared its dirty flags at emission (PipeApply.h:298-304) - so the capacity is
sized so it cannot be reached and reaching it says its own name.
* An answer nobody declared is DROPPED, which is what keeps the store bounded by the
client's deferral depth instead of by the server's answer rate. Retain-on-arrival
grows ~11.3 entries per create against a server that ignores the no-reply bit.
* A read that fails after WaitForApplied has returned Reached is now Fatal{ReplyMissing}
naming the seq: on the stream arm the reply frame strictly precedes the progress
frame that publishes appliedSeq, so that combination is a broken invariant, and
retrying it (what the row used to do) wedges the window for the rest of the
process while leaving every deferred object latched on a provisional accept.
* The residency budget is deleted (WireTables.cpp keeps its history), the window
retirement branch drains once before falling back - it could otherwise strand the
answers it was holding - and Detach clears the store.
MEASURED ON THE DEVICE, load frame, both ends rebuilt, frame-sent identical per op
(create 4,536, records 103,097) throughout:
window 1 (R-5's shape) 4,536 takes 28.66 s
window 2 explicit 2,500 17.27 s
window 2 AS THE SHIPPED DEFAULT 2,563 16.52 s
window 4 1,644 11.02 s
window 2 against an OLD SERVER 2,663 16.38 s (zero Fatals)
The last row is the version-skew case the design review named as a must: a server
built before the no-reply bit answers ~55,903 records a frame, the whitelist drops
every answer nobody declared, and the deferral still reads its own back.
GATES (172 + 15 green), and the important one is proved to go red:
* ADeclaredAnswerSurvivesFarMoreInterveningAnswersThanAnyFixedBuffer - reverting the
store to "keep the last 8 arrivals" fails it immediately with "the declared answer
did not survive 64 intervening ones". That is the line between the ring-1024
stand-in and a bounded mechanism.
* AnAnswerNobodyDeclaredIsNotKeptAtAll - red if everything is retained.
* TheWindowRefusesToDeferOnALinkThatCannotRetainAnswers - on the in-process (ShmLink)
arm the knob at 4 must still block every create, red if the capability gate goes.
* The refused-create case now reads the repair that exists on THAT arm (the latch),
and asserts the window's own books are empty there.
NOT DONE, and named: the window has no unit gate on the STREAM arm yet - only the
device runs. Three rounds of this feature were green in the lanes while inert on the
device because the lanes and the device run different transports; the transport-level
retention gates close that for the store, and the window-level one should be built on
StreamClientPublicationTest's real stream session next.
ILink::ReadReply takes a SEQ (ILink.h:285) and ShmLink implements it against a
ReplySlotPool, so any answer the pool still holds is readable. StreamLink kept ONE
reply - replySeq/replyStatus/reply - and then asked the wrong question of it:
x.replyReady.wait(lock, [&]{ return closed || x.replySeq >= seq || appliedSeq >= seq; });
if (x.replySeq != seq) return ... MOBILEGL_ERR_PROTOCOL_MISMATCH;
The wait is written as "at least this seq"; the check is "exactly this seq". Every
answer older than the newest was therefore unreadable, which nothing noticed because
every caller read the answer it had just waited for - i.e. always the newest one. It
is the transport half of the P12 create window's failure: the window defers up to
four answers and collects them in order, and this link could not hand one back.
THE RING, AT THE POOL'S OWN DEPTH. kReplyRing = kDefaultReplySlotCount (8), because
that is the pool this link stands in for: an answer a reader has not taken by the time
eight newer ones have arrived is one the shm arm has already overwritten too, so the
two transports now lose the same answers and no more. Vectors are sized per reply, so
an idle link pays eight empty ones.
This does NOT make the create window work - on the device it changed nothing, because
the pool itself is 8 slots against 59,673 replies in one frame (see the next commit).
It is kept because the two transports disagreeing about ILink's contract is a bug in
its own right, and because the single slot silently failed any read that raced a newer
reply.
Gate: StreamLinkTest.AnOlderReplyIsStillReadableByItsSeq posts three replies and reads
them oldest-first - the order the window's drain takes them in.
BZLZHH
swung0x48