This is attempt to write down NAME 's proposal to This was surprisingly hard to write, and undoubtedly could be improved. There are cleaner ways to solve this problem, but this one is lowest-impact to the existing spec.
NAME thanks for the new text, that helped. I think we've resolved outstanding issues, unless my new text has created new problems. 2 possible ways to enlarge this solution: should we describe this attack in Security Considerations? As these MAYs are an effort to counter something in particular, perhaps we should discuss. it would clean up some accounting issues if each NCID frame MUST set Retire Prior To to more than the maximum continuous sequence number that it had recieved an RCID for. Worthwhile?
That's interesting to send at first glance, but then poses the question of what the recipient of NCID should do when it sees an RPT value change. There's a difference in behavior between "I've seen these" and "you need to send these."
I think this PR is sufficient to both document the issue and provide some minimum guidance, so I'd suggest we editorialize a bit and then move forward with a solution along these lines.
It seems all the discussion of the problem is happening here and not on the issue, so I will propose my alternative solution here: I know folks have been resistant to changing state based off receiving an acknowledgement of a packet/frame they sent, but what if we changed how the RetirePriorTo works, and just required the packet containing the frame to be acknowledged, instead of requiring the RCID to be sent? Would this solve the problems? This would mean the receiver of the NCID shouldn't have to track any additional state beyond what it takes to acknowledge the packet (which they have to do anyways); it just immediately throws away the state. And the RCID packet would only be used when the endpoint that was given the CID chooses to retire it.
NAME I think this PR has been an attempt to clarify the "leaky" approach, that tries to clarify the amount of state that an endpoint should retain. There are indeed discussion of what that amount is, but I think it is essential to have such detailed discussion on the PR in order to determine exactly what a particular approach would be like. To paraphrase, I think we are making process on this PR to describe a specific approach. If we are to consider something different, I think that discussion should happen on the issue (and emerge as a separate PR).
Ok, trying to put this one in the ground. I think the only remaining dispute is between NAME and NAME on whether activeconnectionid_limit is a sufficient number of RCIDs to remember.
Reading through this proposal, I fear that this would be very difficult to implement. If the ACK for a RETIRECONNECTIONID frame is lost, and endpoint cannot safely send a NEWCONNECTIONID frame with a RetirePriorTo, since this would cause the peer to have more than RETIRECONNECTION_ID frames in flight. In order to safely use RetirePriorTo, this endpoint then would have to track receipt of the ACK frame. I'd very much like to avoid the complexity associated with that.
I finally caught up with this issue, and I think that implementer advice is my preferred way forward. To that end, I like this PR as the resolution.
OK -- there has been an enormous amount of discussion. The current state of the PR is that all RCIDs will eventually be sent, but if there are more than activeconnectionid_limit, some may be delayed by an RTT or more. If it gets out of hand the RCID sender can throw a connection error. I believe this reflects what NAME articulated a few dozen messages ago, and in the thread above I have pseudocode for a lightweight implementation of this (i.e. essentially no additional resources for an arbitrarily large backlog of RCIDs). The advantage of this approach is that it is lightweight and is very subtle change to behavior that will not will not affect interoperability at all or break the existing contract of a Retire Prior To. There are two alternatives that have some traction: 1) Just bite the bullet and do . This breaks interoperability but is a clean architectural change that addresses the root of the problem. I would be very comfortable with this, and prefer it to other things that break the existing semantics. 2) NAME "leaky" approach that allows endpoints to simply not send some RCIDs at all if there are too many pending. I used to prefer this, but given my existence proof above I'm not as excited about it anymore. It breaks the contract that is in draft-27 instead of merely bending it. I hope that covers the decision space. I would like to put this issue out of its misery.
I'm reluctant to blow away the whole PR with brand new text, so in this comment I will propose four entirely new paragraphs that IMO says what needs to be said. Either the current PR, Martin's revision, or this proposal could serve as the basis of the eventual change: If a peer sends large numbers of NEWCONNECTIONID frames that increase Retire Prior To, and/or acks of packets that contain RETIRECONNECTIONID are lost, the state required at the RETIRECONNECTIONID sender can grow without regard to its activeconnectionidlimit. Therefore, endpoints SHOULD take steps to bound the state associated with needed RETIRECONNECTIONID frames while ensuring that it eventually transmits all required RETIRECONNECTIONID frames. For example, it might limit voluntarily retirement of sequence numbers if it has not received enough acknowledgments of packets containing previous retirements. It might also restrict the RETIRECONNECTIONID frames in flight to a single packet in order to simplify tracking of what is in flight, what needs retransmission, and what has been acknowledged. An endpoint MAY treat having too many connection IDs to retire as a connection error of type CONNECTIONIDLIMITERROR. The threshold for this error SHOULD be at least twice the endpoint's advertised activeconnectionidlimit. Endpoints SHOULD NOT issue updates of the Retire Prior To field prior to receiving all of the RETIRECONNECTION_ID frames for the previous update to Retire Prior To. ** If people much prefer this or MT's revision, I'll update the PR accordingly.
I took NAME 's suggested text. I would prefer a little advice on how to implement the lag, but I can live with this text.
LGTM.
When an endpoint sends a RETIRECONNECTIONID frame carrying a connection ID that has been active until that point, that endpoint is expected to open room for accepting a new connection ID. That means that if the peer does not acknowledge the packets that contains RETIRECONNECTIONID frames, but keeps on sending NEWCONNECTIONID frames with increasing values in the Retire Prior To field, the number of unacknowledged RETIRECONNECTIONID frames (or to be precise, the number of unacked retiring connection IDs) keeps on increasing.
To address the issue, I think we need to do one of the following: State that the attack exists, and that QUIC stacks should implement mitigations. Introduce MAXCONNECTIONIDS frame. The reason we do not have similar issue in stream concurrency control is because the credit is controlled by MAX_STREAMS frames, rather than using FIN (or reset) as an implicit signal to indicate availability of new credit. We could do the same for connection IDs.
Mathematically, this might be unbounded, but it's more likely that in practice the limit is as it would be unlikely that unacknowledged RETIRECONNECTIONID frames would remain so once more NEWCONNECTIONID frames start arriving. More so as the issuer of those NEWCONNECTIONID frames has to have received the RETIRECONNECTION_ID frames. Adding new frames for this seems a little unwieldy, so I'd prefer to just acknowledge that the problem exists. Note also that Retire Prior To has the effect of making this limit squishy in other ways. (This is a design issue.)
I'm not sure I understand the attack here. Why would an implementation need to keep track of acknowledgements of RETIRECONNECTIONID frames (other than for purposes of loss recovery, but then this applies to any retransmittable frame type)? An endpoint sends a RETIRECONNECTIONID frame when it decides to not use the respective CID any longer. At this point, it can forget about the connection ID altogether. It might make sense to hold on to the stateless reset token for a while (3 PTO?) longer, in order to be able to detect stateless resets, but this is an optimization and wouldn't lead to unbounded state anyway.
NAME I think is fine, assuming that the endpoint would just stop retransmitting oldest RETIRECONNECTIONID frames. That limit would be too stern to be used for detecting potential attacks (and therefore closing the connection with PROTOCOLVIOLATION).
NAME This is about keeping state for retransmission for recovering loss. As stated in , the issue is unique to RETIRECONNECTIONID frames, because this frame type is (ab)used for providing the peer additional credit, compared to other flow control that uses dedicated frame types (e.g., MAX_*) for communicating credit.
Thank you for the explanation, NAME I think I now understood the problem. This would require endpoints to make assumptions about how the peer handles the and how it issues new CIDs. This can get complicated really quickly and leave the connection in an unrecoverable state (or at the least a state with less CIDs than it should have). Introducing a new frame, as NAME suggested, seems a lot cleaner to me.
Just to be clear, while I used RPT as a way of explaining the design issue, I think it is not a requirement. A malicious client can repeatedly migrate to a new address, initiating the use of new CID pairs, at the same time intentionally not acknowledging packets containing RETIRECONNECTIONID frames sent from the server. If the client does that, the number of CIDs that the server has to track for retirement increases as time goes.
Clarification question: If the person sending RCID isn't getting ACKs back, I think we still need to assume that those CIDs are being replaced? If not, then the non-RPT side of this would just mean the person sending RCID frames runs out of CIDs to use and has to stop. So the problem statement is that: this person is running over their maxactivecid limit from the perspective of the person withholding ACKs, but each side has their own view and so the person retiring them stopped counting them as active once the frame was sent (but not acknowledged). And that can happen via RPT as well, just because it means that they're going to be instructed to send the RCID, but the problem is the same either way? (Just want to make sure I'm understanding the problem correctly)
NAME Yes, I think your problem statement is correct. Or to be even more concise, the problem is that the issuer of CIDs is given additional credit when it receives RCID, which happens before the consumer of CIDs drops the state it needs to retain for retransmitting RCID. For the attack to work, an attacker has to let the peer consume and retire CIDs. RPT and intentional migration are examples that makes that happen.
So the issue here is fundamentally that you can induce the peer to send retransmittable frames which aren't subject to some kind of flow control, and this is the only instance we have of that. STOPSENDING and failing to acknowledge the RESETSTREAM appears to have a similar profile, except that the number of RESETSTREAMs you could have outstanding is bounded by the number of streams the peer has opened, which they control. You could execute the same attack at a larger scale by choosing particular regions of stream data to never acknowledge, forcing the peer to keep them in memory for retransmission, but the peer could stop sending until that region gets through (applying backpressure) or simply reset the stream. Since a RESETSTREAM is smaller than the data region and only one is needed per stream, this collapses into the previous case. What's different here is that the endpoint being attacked doesn't choose whether to retire these CIDs and has no mechanism to apply backpressure. It seems like there are a couple paths to resolving this. As you suggest, we could define an explicit frame that grants the additional CID space, and implementations would not send / could withhold this frame until the RCID is ACK'd. This creates a backpressure mechanism and has the side benefit of allowing an implementation to vary the number of CIDs it wants to track over time, but the disadvantage of requiring an extra round trip before CIDs can be replaced in the normal case. We could mimic the "collapsing" behavior above by defining a frame that retires all CIDs up to a given sequence number. Then the implementation under memory pressure chooses to retire a few extra CIDs and collapses the whole set waiting to be retransmitted to a single frame.
NAME I think your summarization of the problem is accurate. Regarding an attacker selectively acking packets carrying STREAM frames, yes, the attack exists, but it the sender can mitigate that problem in certain ways. We already state that in . This issue is the only case that a protocol cannot be implemented properly, with a bounded amount of state. Regarding how we should fix the problem, if we are going make a protocol change, my preference goes to applying the same design principle that we have in other places (i.e. introduce MAXCONNECTIONSIDS frame). I do not like designing a mechanism specific for this purpose, as there is a chance that we might make mistakes. I can also live with what NAME proposed in URL (i.e. encourage leaky behavior).
Similar to NAME I'd prefer to acknowledge the problem and indicate that peers may want to limit the number of unacknowledged RETIRECONNECTIONID frames to the number of connection IDs they support. The other option I can imagine is to make the RETIRECONNECTIONID frame cumulative, which provides a strong incentive to use connection IDs in order. Though we may have to make an exception for the connection ID in the received packet if we did that.
So the bounds on this aren't that terrible. Say you had to retire 1000 connection IDs all at once. You could track each one individually as you send RETIRECONNECTIONID. That's 1000 things to track, which might be more state than you want (it's not 1000 bits because you need to maintain relationships with packet numbers). Or you could just keep strictly limited, no matter how many connection IDs you need to retire. Tracking that, plus what you have been requested to retire should bound the state required to implement retirement. In other words, you do control how much state you commit to retirement, just like everything else.
NAME That's my preferred solution, but in my reading of the spec this is not allowed. We MUST send RCID for each sequence number for which the peer has sent retire-prior-to. I think the language in Section 5.1.2 needs to change.
NAME idea of allowing RCID to be cumulative is also good and computationally cleaner, although we'd need a different frame type or an additional field in the frame (I don't think we can get rid of retiring non-continuous sequence numbers)
I'm happy to file a PR for the NAME or NAME approach based on the overall sentiment.
One purely implementation option: It's always legitimate to repeat a frame. Therefore, if you track only the smallest and largest un-acked RCID sequence numbers, you can just keep retiring the whole range every time you retransmit. ACKs will shrink the range (hopefully to zero). There's no harm beyond unnecessary bytes in retransmitting something a few extra times to avoid excessive state tracking.
NAME While I won't say that's impossible, I'd argue that the design is going to be complex. When CIDs are used in a legitimate manner, they are not going to be retired one by one. A CID is retired when a path using that CID is abandoned. Consider the case where an endpoint has 4 active CIDs (CID0 to CID3), and two paths. One path could be using CID0, the other path could be using CID3. When the endpoint abandon the path that is using CID3, it has to retire CID3 but it cannot retire others. NAME The amount of CIDs that can be sent on wire at a time is restricted by the CWND, therefore there is no guarantee that you can send RCID frames for all CIDs that you want to retire at once. In other words, I think the approach at best collapses into what NAME has proposed (i.e. leaky behavior).
At this point, I also think NAME solution is the best approach, since it is a very minor change, and changing RETIRECONNECTIONID to be cumulative is a larger change than I want to accept at this point. NAME thanks for your examples.
One idea: Can we simplify everything by not requiring sending RETIRECONNECTIONID for CIDs less than the Retire Prior To? RETIRECONNECTIONID would then only be used when a CID was used and needed to be retired so more CIDs could be issued to the user of the CID.
NAME see my comment in the PR. More disruptive to current implementations, and eliminates a useful signal. The current proposal will not matter for well-behaved clients but will protect servers from some attacks.
We haven't implemented this yet, so it's not disruptive to our implementation FWIW I wonder how useful the signal is. Retire Prior To is really saying "These are going away sometime soon, whether you like it or not." to the peer. If an implementation never sent RETIRECONNECTIONID, would that create a problem of some sort, because I suspect someone will do just that.
NAME As stated in URL, this attack can be mounted without using Retire Prior To.
NAME in this case isn't the client limited by activeconnectionid_limit? As a server, if the client has in no way indicated it has accepted my retirement, I'm not going to accept new CIDs. The NCID/RPT case is different because the spec specifically allows the client to NCID speculatively assuming the server will immediately retire the CID.
NAME The attack here is that when a malicious client retires a CID to which the server has responded, that server would retire the CID that it has used on that path, and also provide a new CID to the client. But the client never ACKs the RCID frames that the server sends. As an example, consider the case where a client uses sCID1 on a new path, the server responds on that path using cCID1, the client abandons that path and sends RCID (seq=sCID1). When receiving this RCID frame, the server would send RCID (seq=cCID1), and also send NCID(sCID2). The client intentionally does not ack the packet carrying these frames, but uses sCID2 on a new path, that carries NCID(cCID2), repeating this procedure.
Ah, I see now that my defense doesn't work, because if I reject the NCID I have to kill the connection, and it may just be because of unlucky ack losses. This whole section is not very precisely written and suffers from two-generals problems about when a CID is actually retired. I will revise to account for this, but I'm not sure how.
NAME I agree with NAME that there's no valid thread model without Retire Prior To. If the peer doesn't acknowledge your retirement, then you shouldn't be issuing more CIDs, because you might exceed their limit.
In my opinion, running out of CIDs is not a big crisis. You can always create a new connection with 0-RTT with a relatively small overhead. We should be biased towards simplicity rather than never running out of CIDs.
NAME unfortunately we are cornered by the spec. Consider two well behaving endpoints. I'm fully stocked with CIDs and then retire one with a frame. The peer gets the RCID and issues an NCID, but the ack of RCID is lost. If I don't count the CID as retired, then I MUST close the connection. That's bad, so I have to treat the CID as retired if I sent RCID. If you drop acks at scale, this becomes an attack. I'm really coming around to your cumulative retire idea, but will have another go at this tomorrow.
NAME I think the framing is incorrect. The peer can enforce an endpoint to retire a CID (e.g., by changing RPT or discarding a path). As NAME points out, an endpoint is encouraged issue a new CID at this point, rather than withholding than until the ACK of RCID is received. Note also that RCID that an endpoint sends carries the CID that the peer has issued, while NCID that the endpoint carries the CID that that endpoint issues. Therefore, I am not sure what the rationale for withholding the emission of NCID until receiving an ACK for RCID would be. I agree. An endpoint is not expected to consume CID at high rate. We already discourage such behavior; see URL That's why I'm fine with the leaky behavior that NAME has suggested. NAME As previously stated, I am hesitant to have a new machinery that is unique to this purpose. There is a risk of creating something buggy, both in terms of specification and implementation. My preference goes to accepting the risk (and recommending mitigations, such as the leaky behavior), or reusing a design pattern that we already have (i.e. introduce MAXCONNECTIONIDS frame).
The cumulative idea is very appealing to me, minus a future with multipath. If two or more paths are active at once, I don't know how the peer would know that if the frame was cumulative. The only option would be to ensure the two(or more) CIDs in use are adjacent, which is possible, but may require switching CIDs more than should be necessary.
NAME I would suggest a cumulative RCID as a supplement to the individual RCID. A cumulative one would cleanly handle a lot of cases where there is an unboundedly large list of sequence numbers to retire. If there is a low sequence number that is still in use, the sender can choose to retire it to enable use of the cumulative frame.
I know folks have been resistant to changing state based off receiving an acknowledgement of a packet/frame they sent, but what if we changed how the RetirePriorTo works, and just required the packet containing the frame to be acknowledged, instead of requiring the RCID to be sent? Would this solve the problems? This would mean the receiver of the NCID shouldn't have to track any additional state beyond what it takes to acknowledge the packet (which they have to do anyways); it just immediately throws away the state. And the RCID packet would only be used when the endpoint that was given the CID chooses to retire it.
FWIW, I have filed that fixes this issue without introducing a leaky behavior like . IMO, the observation is that the endpoints can check that the issuer of CIDs is not intentionally introducing gaps in the sequence numbers they generate. We already prohibit senders from introducing such gaps, detecting them on the receiver side would be a good thing to do. And if endpoints start doing that, using a cumulative RCID frame is a sensible solution, as the the cumulative list of retired CIDs at any time can be represented using a SACK-like structure (i.e., max-retired + small list of active CIDs below that max.).
I know that you have characterized this as "leaky", but I'm not sure that I understand exactly what is being leaked. Nothing is lost in , except that the endpoint requesting retirement has to tolerate a potentially extended period without old connection IDs being retired. In case this helps, I have no big preference between and . There is a certain elegance to that might tip me toward that, but it is slightly more disruptive. I prefer either of those to the more disruptive change in , which also has the downside of strongly encouraging connection ID changes in order to preserve a clean separation between active and retired. doesn't force you to drop old connection IDs. However, the encoding of the frame is just too complicated for me to accept. I would prefer forcing a clean separation than having a complex encoding like this. is a non-starter for me for the reasons stated.
NAME In the approach we take in , there is a possibility that the issuer of the CID might not receive the sequence numbers of all the CIDs that have been retired by the peer, when the issuer is issuing new CIDs at high rate and the receiver is also consuming them at a hight rate. That is the consequence of endpoints limiting the amount of inflight RCID frames it tracks. I think we can provide advice so that the "leak" would not be an issue in practice, and assuming that would happen, I am happy with being the solution. I think is cleaner, but I acknowledge that it is more disruptive than . Part of my intent behind creating is to see how a clean fix (in terms of state machinery) would look like, so that it can be compared against . As stated in URL, by itself only partly fixes the problem. I am not against adopting along with though. is also a partial fix, I also think that it is a non-starter.
I think that I understand your point; thanks for your patience. I didn't really classify it as a leak though, so the choice of words threw me a little, it's more of a lag. Let's say that you have a pathological endpoint that sends an infinite series of NEWCONNECTIONID frames with Sequence Number N and Retire Prior To of (N-1). That's legal and will never violate any limits. But unless they maintain acknowledgments for RETIRECONNECTIONID frames at a similar rate (which, to be fair should be easy as for the same values), they could get far ahead of their peer. I don't think that it is strictly a leak, just a lag between the connection IDs being pushed and the connection IDs being successfully retired. Without or something else this could be problematic. And I now understand why is really just an orthogonal refinement, though it might make other defenses less likely to be required by making zero in cases where an endpoint isn't using discretionary retirement. The defense in is to make effectively constant, no matter how many frames need to be retired. My main objection is really that it is a maximal design where only a minimal one is warranted. It's probably worth pointing out that there is also the simplest defense: If you can't keep up with RETIRECONNECTIONID, stop talking to the peer that does this to you. This is just another example from that category of abuse we don't have specific protocol mechanisms to mitigate.
I agree that I'd like to see a minimal solution, so I'm leery of heading towards . In order for a peer to need to send a lot of RETIRECONNECTIONID frames without Retire Prior To, a lot of 5-tuple + CID changes need to be coming in. One way to rate limit that naturally is to stop giving out CIDs if the peer hasn't acknowledged your RETIRECONNECTIONID frames. ie: Limit the number of NEWCONNECTIONID frames in flight to: So if the peer stops acknowledging your retirements, you stop giving them new CIDs. I think this avoids the need for any other limits? I added 1 in my example so one NCID could be sent even if all of the peer's CIDs had just been retired, though that concern disappears if we decided to also do something like
What a mess we've made! Is there interest in doing a Zoom about this? Maybe 0900 Tokyo/1100 Sydney/1700 Seattle/2000 Boston on Tue/Wed? The root cause is that the flow control mechanism isn't all that well designed. IMO is the actual correct fix to this problem, but as people say, it's disruptive. uses ACKs as an implicit signal, which seems like a problem. has a different implicit signal, and I don't think it solves the case where the client is withholding acks for unsolicited RCIDs. I really think we should just use or depending on our appetite for changing the wire image. is great for the RCID sender, and if the receiver won't miss any RCIDs unless it's doing pathological stuff or the ack loss patterns are tremendously unlucky. , OTOH, is a comprehensive fix to the problem.
I think with some added text about limiting outstanding NCIDs when there are lots of RCIDs would be sufficiently robust.
I'm fine with trying to see if we can reach an agreement on .
I'd prefer over . I can't really see why it would more disruptive than . The problem with is that it will likely work find in most cases, but there are some corner case (when the ACKs for RETIRECONNECTIONID frames are lost), in which things will break. Properly accounting for these corner cases requires some non-trivial tracking logic in order to avoid risking the connection being closed with a CONNECTIONIDLIMIT_ERROR. On the other hand, seems to be conceptually simpler. I'd rather implement a few changes to my frame parser than introduce some ACK-tracking logic.
Just to give another example of how an endpoint might end up required to track the retirement of more than activeconnectionid_limit, when talking to a well-behaving peer: A server tries to supply three active CIDs to the peer. At the moment, it has provided CID0, CID1, CID2 to the client. The client sends RCID(CID1), RCID(CID2). The server provides NCID(CID3), NCID(CID4). The two NCID frames are delivered to the client, but the ACK for the RCID frames gets lost. The client decides to probe two paths concurrently, by using CID3 and CID4, but shortly after the packets are sent using those paths, the underlying network for those two paths disappear. The client now needs to make sure that RCID frames carrying 4 CIDs reach the peer. In this example, we might argue that it would be possible for the client to figure out that CID3 and CID4 were provided as substitutions for CID1 and CID2. But do we want to implement logic that detects such condition into our code? Note that tracking of substitutions can become tricky; if RCID(CID1) and RCID(CID2) were sent in different packets, it would be impossible for the client to see if CID4 was provided as a supplement for CID1 or for CID2.
I agree with NAME that if we go with the limit should be higher than .
I finally caught up with this issue, and implementer advice is definitely my preferred way forward. We're going to need some complexity to (and some magic numbers) no matter what, let's do this while limiting mechanism to a minimum. To that end, I agree with as a good resolution.
While increasing the limit will certainly help, this introduces a really unpleasant flakiness in the protocol. QUIC, so far, has the property that no amount or pattern of packet loss will lead to a protocol violation (it might lead to a connection timeout if all packets are blackholed, but that's a different kind of error). This means that when running QUIC in production, screening for (transport-level) errors will be very useful for finding bugs in implementations. My fear regarding is that no matter how high we choose the limit, there will always be a loss pattern that will trigger a protocol violation.
NAME Thank you for raising the concern, I think the recommended behavior when exceeding that limit is to stop sending RCID frames for some sequence numbers rather than triggering a protocol violation. That's actually what I had in mind hence called it "leaky." The benefit of stopping retiring some sequence numbers is that the worst case becomes endpoints running out of CIDs - something that is expected to happen on ordinary connections too. Note also that in HTTP/3, reestablishing a connection in 0-RTT is always a viable option. To summarize, I think that the concern can be resolved for HTTP/3 by changing the recommended behavior from resetting the connection to stop sending RCIF frames for some sequence numbers.
NAME I don't think not sending RCID frames is that easy. Tracking something that needs to be sent consumes comparable state to tracking it in flight. Or are you suggesting a different algorithm that indicates a RCID frame doesn't need to be sent? If the peer has to wait for one Retire Prior To to take effect before it's increased again, that leaves only the peer migrating really quickly and lost ACKs as the unbounded case. As NAME said, I think it's more likely a well-behaved peer would run out of CIDs in this case than any other failure mode. I previously suggested stopping sending out NCID frames when the RCID frame limit had been reached to force the issue, but that just changes who has to close the connection. FWIW, we already have sanity checks where our implementation closes the connection when a datastructure becomes too large. The limits are high enough that they're almost never hit unless there is a bug, but they've been helpful in finding bugs. I would advocate everyone have these. NAME You'll never get to 0 transport errors. Some implementations will have a bug, and some peers or servers will have faulty or corrupted memory. For example, in Google QUIC we found we received the ClientHello on a stream besides 1 surprisingly often.
I think that the idea is to reduce the set of RCID frames needed to a range of sequence numbers. You only have to remember the few exceptions for sequence numbers that are in use, sequence numbers for which you have outstanding frames, and a range of sequence numbers that require retirement. As long as your discretionary use is sequential, this is bounded state. As mentioned, the risk is that lack of acknowledgment leads to potentially large delays in getting new connection IDs, but if you ever run short, abandon the connection. Either you have a pathological loss pattern or your peer is messing with you. In either case, it is probably not worth continuing.
NAME Yes, this is one method of limiting the state that is being used for retaining sequence numbers that need to be sent. And it works even when discretionary use is not sequential, as the number of "exceptions" (i.e. gaps) is bounded by maxconnectionidlimit. The other assumption of using this method is that you would somehow bound the state that is used to track RCID frames being inflight. There are various ways of accomplishing that, e.g., have a limit on the number of RCID frames sent at once, track bunch of inflight RCID frames as one meta-frame. And it is correct that the outcome would be a "lag" in this method. The downside of this method is that it might be a disruptive change to existing implementations. It is essentially equivalent to what proposes, with the exception being that tracking RCID frames are difficult (as you track to send many). For existing implementations, it would be easier to adopt a "leaky" method, i.e.: let endpoints have a counter counting on the number of sequence numbers that are retired but are yet to be acknowledged when a CID is being retired, if the value of the counter is below a certain threshold (i.e. 2 maxconnectionidlimit), the counter is incremented, and a RCID frame carrying the sequence number of that CID is added to the outstanding queue. If the counter was not below the threshold, no RCID frame will be sent (leak!) when an RCID frame is deemed lost, it will be resubmitted into the outstanding queue when an RCID frame is acknowledged before deemed lost, the counter is decremented when an RCID frame is late-acked, we do nothing Assuming that the solution we prefer is the least-disruptive approach, I think we need to recommend the latter. If we have distaste in the latter and prefer the former, I think we'd better switch to , as it more cleanly reflects the design of the former. NAME I think the latter is not that a big change to existing implementations.
Yep, I think that is a good approach. Obviously, the range encoding is superior in terms of how much state it can hold, but a simple list is fine. And the effects of lag will be limited by the tolerance for retaining retired connection IDs, which seems proportional to me.
NAME I doubt that's less impactful on existing implementations, certainly from an interoperability standpoint. Today endpoints should hold on to incoming CIDs until they are formally retired. Making them leaky can lead to trapped state. Sending them all, but with a delay, is no added state but solves this problem.
After a lot of back-and-forth, I think that we've all generally converged on .
Still LG, but I like ekr's suggestionLatest text looks good


Behold
NAME I think this is inches of getting landed
The full text of that section is "TBD."
HPACK covers the following topics: Probing Dynamic Table State Static Huffman Encoding Memory Consumption/DoS Implementation Limits To what degree to we need to retread this vs referring to 7541? There are some new areas to cover related to DoS, like recommending implementation timeouts for blocking requests. Soliciting any other topics, and possibly a passionate security-minded volunteer to draft a PR.
All looks good (and familiar); some minor nits.


This explains what needs to be kept and why. Specifically, you need to keep ranges unless you have other means of ensuring that you don't accept packets from those ranges again. You also need to keep the largest acknowledged so that you can get a packet number from subsequent packets. This also recommends that ACK frames include the largest acknowledged always. That is primarily to ensure that ECN works properly, and even there, you only disable ECN if you get some weird reordering, so it's probably not a big deal if you don't follow this recommendation. The issue is marked design, but the resolution here is basically a restatement of other text. I think we can run this through the design process, but it isn't really worth flagging this in a change log in my opinion.
NAME you should probably look at this. I have marked the associated issue anyway, but I'd appreciate another review.
In , NAME notes that added this "MUST": My understanding is that this prevents the largest acknowledged packet from going backwards. We use the monotonic increase of the Largest Acknowledged field in two ways: In ECN validation, this is used to filter out ACK frames that might have arrived out of order, which might have lower counts legitimately. In key updates, where the value of the largest acknowledged is used to drive the change to new keys and to prevent use of old keys after an update. In my view, having ECN validation fail is quite unfortunate, but likely workable. On the other hand, uncertainty about largest acknowledged could introduce instability in the key update process. If an endpoint's conception of what the value is changes, I doubt that we'll see extra key updates (I think that the AEAD would protect us from that), but we might leave open the possibility of interleaving of packets from two different key phases. I would suggest that we retain this "MUST".
For out of order ACK frames, you can use the packet number the ACK arrives in. I realize that we have tried to allow frames to be retransmitted, but I think retransmitting ACK frames without modifying the ACK delay/etc is so harmful we should probably prohibit that one frame from being retransmitted, since it creates a bunch of edge cases. Or we could let ECN validation could fail, but that'd be a bit sad. I'm confused about what the issue with key updates is? The largest acked you've ever seen doesn't change, just the largest acked from the last ACK frame. When determining the size of the packet number, we say: "The sender MUST use a packet number size able to represent more than twice as large a range than the difference between the largest acknowledged packet and packet number being sent." I'd rather not make this a MUST unless we have a good reason. There are valid use cases that become more complex with this constraint.
I really want to understand the use cases that you think that this might break. Because key updates very much depend on tracking largest acknowledged. The key updates case has a fairly simple example. Say you receive packet 10 in key phase 1. Then 11 in key phase 2. That's valid. But if you ACK those and then only remember up to packet 8, packet 9 can use key phase 2 and you won't know that it is invalid.
I'd like to suggest tracking largest acknowledged from forcing all ACK frames to contain the largest acknowledged and have that value never decrease. Given you need to track largest acknowledged locally(ie: not only in the in-flight ACK frame), I don't think there are any problems there. I have two cases in mind where the largest acked could validly decrease: 1) A receiver that tracks a limited number of ACK blocks and the oldest block happens to be the one containing largest acked. 2) If an application was reordering tolerant(ie: from URL) then two different senders on a single 'connection' may be sending packets with disparate packet numbers, causing what appears to be very large scale reordering. In this case, it'd be valuable to discard the older ACK ranges even if they contained the largest acked. Maybe the first is an edge case we don't care about in a use case we aren't optimizing for and the latter is solvable with careful issuing of packet number ranges. Even so, I'm not sure what the motivation is for this change? I guess the ECN case could be, but using largest acked as a proxy for retransmitted ACKs is a very bad design in my opinion.
I don't think we really need to cater for retransmitted ACK frames in any way. What I'm concerned about is reordering. Yes, using the increase of largest acknowledged as a proxy for strict ordering is a little weird, but the assumption is that frame handling is distinct from packet handling and packet numbers are not available when frames are handled. (I think that's the principle we agreed on; that's why it's OK to use packet numbers in key updates, for example, something that I can confirm is borne out by implementation experience.) The case you describe here has the appearance of exactly that type of reordering if the two senders also generate their own ACK frames (though this seems very strange). But I think that the consequences of that sort of design are not our responsibility to deal with.
I don't recall on us agreeing that packet numbers weren't available when frames are handled, can you clarify why that property is important?
I agree with NAME on the principle of not requiring the enclosing packet's packet number during frame handling, and we use sequence numbers within frames to help with reordering. This was at least true in an earlier version of NCID frames, and we had discussed this principle then. This is also why the ACK_FREQUENCY frame in URL includes a sequence number. That said, I don't understand the issue with key updates here. Largest acked so far is a connection-level state variable that can be used for key updates. I suspect that I'm not understanding the example scenario. NAME can you elaborate on your key update example? NAME Is your example motivated by a receiver that does not remember the largest acked or one that does not wish to ack anything but the packet that was just received?
I was under the impression that the argument was that largest acknowledged was not being tracked, but Ian made it clear that the concern was about not signaling it. That removes key updates as a concern. If remains unclear, then I can't really do better.
NAME my example was motivated by either case, but primarily not wanting to ACK anything besides what was just received.
Perhaps it's a naive model, but I had envisioned an implementation remembering a chain of ranges of packet numbers it had received or not. The implementation would save space by forgetting the earliest packet numbers when the chain got too long, and would construct an ACK frame out of the latest set of these ranges it had room for. In that mental model, at least, there's not really room for the idea that you might drop the largest-acked frame number from your memory because it's too old -- it's always at the most-recent end of your chain, even if you currently happen to be receiving packets that fill in the middle for some implementation-specific reason. (Implementation-specific to the peer, which you obviously can't know about or code for!)
That matches my model for this. It is also likely the case that the next packet you receive is largest+1, so there is no real advantage to forgetting the largest acknowledged. Not sending largest acknowledged is slightly different though. I can understand how an implementation might choose to send only ranges that have recently changed. My contention is that this is not a good choice and we should recommend against that.
NAME to produce some text here that separates retention policy (MUST) from signaling policy (SHOULD).


We don't ever really say that QUIC depends on TLS providing the properties that we claim TLS provides. So it's worth pointing that exposure. This is more so because some of the properties we claim are dependent on these properties.
(Cleaned up duplicate instances of the same comment.)
As per comment URL I'll note that there is MITM protection in QUIC, but it is not immediate which is roughly why we have the concept of handshake confirmed.
The options for triage are close it, mark as editorial, or mark as design. This has a needs-discussion, and I'm seeing a lack of any discussion. So you might see where my thought process is going... NAME this relates to your PR on the QUIC threat model. Are you willing to provide some comment that helps us triage the issue?
I think that we can take this as editorial, as it merely requires that we capture already agreed principles (we don't protect against an adversary who can inject packets). NAME assistance would be useful, but I can produce text if nothing is forthcoming.
I can always provide comments, no guarantees on how much they'll help triage IIRC, this is an issue filed from a comment on the threat model PR that was several-months merged to potentially increase the scope of what that covered. I think that the original threat model that NAME and I put together, covering the handshake and migration respectively, is intended to be the start of that section -- as additional items are discovered that could use coverage there to help clarify QUIC's stance on whatever issue, they should totally be added. I agree with NAME that adding such items seem editorial, as they aren't introducing any new requirements and are only capturing current realities as specified elsewhere. (Of course, as that text is written, it seems logical that putting it all in words next to each other may highlight issues, and if the WG decides that we don't like how the current spec results in a particular security stance, we can open a design issue to change normative text elsewhere.) So, I'd probably consider this editorial, and ask NAME to potentially contribute some text or clarify what he's looking to have covered by the original comment -- once that's clear, happy to help get a PR up if needed (or NAME text is always good too!).
Since this concerns the handshake, I'm not too confident in contributing text but I'd be happy to review and discuss. I believe it is correct that the issue is covered in existing text elsewhere but as NAME says, it makes sense to cover it in one place. The risk is that the reader is not aware how important it is to reach a certain point in the connection before trusting data, and especially to those seeking to create derived versions. From various discussions I'm not sure the importance is broadly understood. Additionally, I'm not entirely convinced that the current text fully captures the problem - not that we necessarily can - for example various stateless transmissions / redirects / token issues - might assume a level of privacy that doesn't exist in all cases, thus requiring stronger integrity elsewhere such as in CIDs, tokens etc..
Thanks folks. It seems we have a path forward, so I'm removing the needs-discussion label. The editorial process will naturally generate some back and forth but that is the same for all editorial issues.


Clarify reserved values for stream types, settings identifiers, frame types, and error codes. 0x1f N + 0x21 for the value of N = -1 is 0x02. Literal interpretation of the current text includes this as a reserved value. The parenthetical examples given in Section 11.2 make it clear that this was not the intention, rendering this PR editorial. The main motivation of this PR is not that the current text is inconsistent, but that parenthetical examples from 11.2 are necessary to correctly interpret the definitions in sections 6.2.3, 7.2.4.1, 7.2.8, and 8.1. Alternative wordings could be: "0x1f N + 0x21 for N = 0, 1, 2, ..." "0x1f N + 0x02 for positive integer values of N" "0x1f N + 0x02 for N = 1, 2, 3, ..." "0x21, 0x21 + 0x1f, 0x21 + 2 * 0x1f, ..." "a value of at least 0x21 with a remainder of 0x02 modulo 0x1f" "a value of at least 0x21 that is congruent to 0x21 modulo 0x1f" none of which is better than what this PR proposes.


This defines a limit on the number of packets that can fail authentication before you have to use new keys. There is a big hole here in that AES-CCM (that is, the AEAD based on CBC-MAC) is currently permitted, but we have no analysis to support either the confidentiality limits in TLS 1.3 or the integrity limits in this document. It is probably OK, but that is not the standard we apply here. So this might have to remain open until we get some sort of resolution on that issue. My initial opinion is to cut CCM from the draft until/unless an analysis is produced.
In the interest of transparency, I have added one more commit here. When I did the same for DTLS, ekr observed that if you have updated, you might safely stop trying to accept packets with the old keys rather than killing the connection. That results in loss for any packets that genuinely did want to use the exhausted keys, but that is probably less disruptive than having the connection drop. Of course, you can't always update, so closure is still likely necessary in some cases.
For what it's worth, Martin, Felix, and I put together a document which might help users do this themselves: URL
NAME Could you use that to come up with 2^16 limits and put them in the doc? I’m really wary of letting implementors roll their own
Sure! Can you please file an issue with that request? Noting anything else you'd like spelled out in detail would be helpful, too.
tl;dr We need to recommend limits on the number of failed decryptions. At one of the side discussions concentrated on the quality of the analysis of the and the applicability of that analysis to QUIC. Felix Günther, along with Marc Fischlin, Christian Janson, and Kenny Paterson have done a little analysis, based on and the we used to set limits in TLS. There are two relevant limits in this analysis: The confidentiality bound, which is usually expressed in terms of advantage an attacker gains in being able to break confidentiality of records/packets as they get more data from valid packets. For instance, in AES-GCM, the analysis shows that after seeing 2^24.5 packets (almost 24 million) the attacker has a 2^-60 better chance than pure chance of winning the IND-CPA game. The forgery bound, which is again expressed as an probably of an attacker successfully creating a forged record/packet. Thanks to the analysis done previously, we know that AES-GCM and ChaCha20+Poly1305 have a forgery chance of 2^-57 after 2^60 and 2^36 forgery attempts respectively. The current text just points to TLS 1.3, which recommends the use of key updates to protect confidentiality. But TLS has a forgery tolerance of 0: a single failed forgery attempt causes the connection to break. QUIC (and DTLS) allow for multiple forgeries to be ignored, so we need to factor that in. As discussed in if we want to maintain security, we need to consider both confidentiality and integrity. For that we need two independent limits: The number of packets that are successfully encrypted and decrypted. The number of packets that are unsuccessfully decrypted. We have the former already. The latter is what this issue tracks. Once the limit is at risk of being exceeded, updating keys will force the attacker to start over. Based on Felix's recommendation I am going to suggest that we limit forgery attempts to 2^36. I have confirmed this with a close reading of the paper and concur. There is a significant difference here between AES-GCM and ChaCha20+Poly1305 but we believe that setting a single limit is the best approach. I will create a pull request to document this approach shortly. We could split the recommendation and allow a larger limit for AES-GCM, but I don't believe that 2^36 (~68 billion) is a significant barrier. For reference, if you were able to saturate a 10Gbps link with 50 byte IP packets that contained plausible QUIC packets, and the recipient didn't treat this as DoS traffic, it would take 5 minutes. If we assume that are more likely representative of potential packet rates, then the limit goes up to 10s of hours. I would point out that that volume of unproductive traffic is probably a good reason to use something other than a key update to deal with the sender. A reminder that the underlying analysis assumes that packets are all 2^14 in size, which is the record size in TLS. This is ~10 times larger than typical MTU on the Internet and many QUIC packets might be smaller than that, so - at least for AES-GCM - recommending a 2^36 limit is more conservative than necessary. However, if we were to allow for different limits, we'd have to address the question of packet size, as a 2^16 byte packet will be used by some deployments. For ChaCha20+Poly1305, the only value that matters is the number of packets, so we avoid that question. Quoting from Felix's email for reference (with permission):
NAME Can I request that you review this analysis for correctness? I know that you had reservations about the claims here.
Another (simplier?) option could be to just update keys much more often. I've done performance tests with msquic on the effect of updating the key ~once per round trip. There was no noticeable effect on performance. In fact, more often than not, the key update perf tests happen to perform slightly better (within noise tolerance) than without key update. I'm not suggesting we advocate once a round trip or anything near that; more like every 2^20 packets (or 2^30 bytes?) sent or received (including failed decryptions). As I understand it, this is an ultra conservative number as far as protecting from any kind of attacks, and should be free as far as any performance impact on the connection. This also has the added benefit of exercising the key update scenario more often, which should hopefully improve interoperability of the feature.
I support a change like this, but I am not qualified to say what the thresholds should be. There certainly isn't a reason to push it to very high limits. This change would not break interoperability, so it's a good one. Banning CCM might break interoperability for someone, but I'm not too concerned, particularly if we can update all the browsers to not offer it. A somewhat bigger concern is the lack of key update support in a lot of implementations: I read 9 of 15 servers as supporting it.
(Tentative) good news on CCM: Kenny identified a security analysis paper and it seems relatively simple to go from that to numbers. I am not personally concerned about support for key update. 9/15 isn't 15, but that doesn't bother me. It turns out to be a little fiddly to implement, so if people decide to drop connections that hit these limits instead, the effect isn't that bad. Of course, if you want to follow Nick's suggestion, that works too (I remember Dan Bernstein seriously suggesting a cipher that updated every record, back when I first proposed this scheme for TLS).
If we're looking for a new bound to initiate a key update, there's precedent in the SSH RFC to rekey after every 1 GB of data transferred. See
I should point out that my "9 of 15" figure is based on the interop matrix, where the client initiates an early update and the server responds. That is a different thing than a server initiating an update when required by the spec, which fewer than 9 servers might do.
Based on by Jonsson, I have calculated the bounds on the number of forgeries and encryptions for CCM. The numbers aren't great, but that probably doesn't matter much for the purposes to which CCM is generally put. (I'm using the notation from the here because that is easier to type than some of what is being used in other papers. For reference, q is the number of genuine encryptions, l is the length of each in 16 byte blocks, v is the number of forgeries, n is the key length in bits, t is the tag length in bits.) Jonsson puts the advantage an attacker has over a generic PRF at . If we assume the same numbers in the AEBounds paper for record/packet size, (this is for consistency, most QUIC packets will be much smaller, in the order of 2^7, but we might also go as high as 2^12; this being dictated by the MTU). For AES, . To keep the advantage an attacker has to 2^-60 (to match the analysis in TLS), we therefore need to keep q to below 2^24.5. Somewhat unsurprisingly, that matches the numbers we have for AES-GCM. Jonsson puts the advantage for an attacker over a generic PRF at . As the first term is negligible even for large v (up to 2^64), we consider the second term alone and aim for a bound on the advantage of 2^-57 to match the analysis for other ciphers. That leaves us with . As q is already established to be 2^24.5, we can say that v should be limited to 2^25 (or if you want to be precise). (As a side note, for t=64, as in AEADAES128CCM8, the first term becomes relevant and our security bound limits the number of forgeries to 2^7, which is probably a bit limiting in practice. That's good justification for not enabling AEADAES128CCM_8 by default; though different applications might use a different target bound than 2^-57.) I'm an not a cryptographer. These papers all read like chicken-chicken-chicken. My calculations are not infallible either.
The shape of the bound seems plausible but I'll start a thread with the cryptographers to check everything
I reproduced Martin's analysis of CCM based on of the bounds by . (Myself not being an expert in producing such bounds, I found the second-level confirmation by Rogaway to be helpful.) I essentially agree with the numbers, except that --when following the same metric as the -- the advantage over an ideal PRP (instead of PRF) should be considered. This simply means losing a 1/2 factor in the bound: Confidentiality bound: yields an advantage of . I'd hence say it's fine to keep TLS 1.3's recommendation on sending max. full-sized records. Integrity bound: , hence yields an advantage of (matching the other analyses). Most importantly, I concur with Martin that AEADAES128CCM8 would need either much lower bounds or a different risk assessment: applying the same limit of would bring the advantage incurred through the bound up to . Would it still be the best approach to define the same anti-forgery limit across all cipher suites, even if this means going down to ? Of course, I'm happy to discuss more with NAME !
Thanks for checking this NAME I was a little concerned about the PRP/PRF split, so I'm glad to have you correct me. I ultimately went with per-AEAD recommendations, and a lower limit for CCM. That's mainly to establish some sort of uniformity around 2^-60 and 2^-57. Even if those are basically arbitrary choices, at least they are defensible (I think!) and using them uniformly establishes the right expectations about what the standard is. I'm comfortable with specifying different limits for each AEAD in specifications. In practice, however, I expect that a far lower tolerance for forgery attempts. Assuming that these numbers work out, a review of the pull request would be greatly appreciated. If there are other relevant papers, I'm always happy to pull those in as well.
I realize that I didn't respond to NAME That advice applies to the limits that TLS specifies already; this issue is primarily about responding to forgeries. That said, 1G is pretty good advice. However what we've seen from this analysis that 1G results in a lower safety margin than TLS aims for. But it's close enough that if the AEAD is as good as the set we have here, you are probably OK. However, this does depend a lot on the specific AEAD. I'd be uncomfortable using so simple a guide in the general case.
NAME points out an error in my calculation. I had based my calculations on direct mapping of in the CCM analysis to in our calculations. That is, was the number of blocks in the message. But the analysis by Jonsson defines this as: ! The definition of Beta is the CCM function, so this function really reduces to 2 times the length of the message (in blocks), plus 1 (to account for the additional encryption). We can ignore the extra 1 as that is absorbed by the tag length. I'm updating the numbers in the PR. The result is another halving of the number of packets. NAME NAME it's pretty clear that I'm out of my depth here, so your input would be highly valued.
Good catch, yes, the double passing over the message requires a factor 2 here. (The AD a may also be up to 3 blocks if I understand the header format correctly, but I guess that's negligible compared to the conservative 2^10 message size.)
Good point about the AD. The AD - at least for QUIC - is always sent, so that fits in the same space. We don't necessarily need to double count the tiny number of blocks, but as you say, counting that only once yields a negligible difference. If we had a large AD that wasn't also transmitted, that would be different.
Agreed. To be precise: if I'm not mistaken the AD isn't counted at all right now ( accounts for the max. payload blocks), but of course the difference is still negligible as long as the AD is small.
Thanks for the edits!


Any chance we could also update the initial salt?
I was saving that for the final version, by popular demand. If you have a reason to change, then I can do that (I'll also want to update the keys for Retry in that case).
I agree with NAME Let's leave the salt update to the final version.
With Google's QUIC deployment, we're already experiencing version ossification due to middleboxes inspecting initial packets - some of our traffic is stuck on older versions of GoogleQUIC because of this. Changing the salts every time we change the version number will help remind everyone that this is not an ossified aspect of the protocol. The cost to implementors is pretty much zero. NAME out of curiosity, why the pushback?
What NAME says makes sense to me, so I would be in favor of shipping draft-29 with new salts.
I updated the salts and the test vectors in quic-go, and I can confirm that the values match.
The draft describes that a client SHOULD NOT reuse a token in different connections (URL). However, I miss the argument that the server SHOULD NOT construct the same token multiple times. This argument addresses the risks of client tracking via tokens if a deterministic approach is used to construct them.
Maybe ?
NAME do you mean already covers this?
Yes, my view is that and the corresponding mailing list thread (URL) covers this issue. They discuss about unlinkability between the tokens issued by a server.
NAME do you agree that this is (mostly) covered by ? If yes, can we close this issue and resolve it as part of ?
No, does not cover this particular point. makes the point that a network observer should not be able to derive information such as a timestamp from observed tokens. However, this issue says that the construction of the token should not be deterministic. To illustrate this problem, assume that the token is more or less the hash of the client's source address. Thus, the client receives upon repeat connections each time the same token, if its source address is identical. As a result, even if the client uses each token only a single time the corresponding connections established with these tokens can be correlated to each other by a network observer.
NAME could be modified to cover your issue? could you comment there?
NAME Ok, I will make proposals within to cover this issue.
NAME the automation failed here - this should not be moved to "Text Incorporated", it should just be closed As stated in URL, my understanding is that a token sent in a NEW_TOKEN frame SHOULD be encrypted. The only exception would be the case of the token carrying only an opaque and randomly-generated key identifier in a stateful design (i.e. a server using a key-value store to retain the information of previous connections). Otherwise, an observer can use the information contained in the token sent in an resuming connection, to correlate that connection to the previous connection that issued the token. For example, if a server includes the generation time of the token in plaintext, an observer can extract that information found in an Initial packet to nail down the connection that issued the token. While I think we might be call this an editorial issue, I think we should clarify this as a normative requirement (e.g., SHOULD). Hence bringing the discussion to the mailing list as well. Kazuho Oku


When we prohibited the use of application-layer CONNECTION_CLOSE in these packets, we should have updated the text in the description of the packets also.
I think the spec is ambiguous when it says in section 17.2: and: >Handshake packets MAY contain CONNECTIONCLOSE frames. But CONNECTIONCLOSE can mean 0x1c or 0x1d. Are both permitted during the handshake?
NAME In this case, the intent is to allow both. Unless explicitly specified, I would read the text to mean both types of CONNECTION_CLOSE frames.
Okay, if more people think that way, we can close with no action.
So on the surface, there is no inconsistency: the two sections permit CONNECTIONCLOSE. But the question about the different types of CONNECTIONCLOSE is more relevant than NAME response might imply. We currently require that CONNECTIONCLOSE of type 0x1d (application close) is not sent in Initial or Handshake packets. So that's a drafting error, and we might reasonably take this as an editorial issue. However, if the question is about what a reaction to receiving an application CONNECTIONCLOSE in one of those packets, we might consider a different posture. If you get an application CONNECTIONCLOSE in an Initial packet that is otherwise valid, you probably want to drop the connection. And you probably don't want to start sending CONNECTIONCLOSE in response every time, because that leads to madness. But this is a general problem: if you detect and error and that error keeps being repeated, it probably isn't a good idea to generate a CONNECTION_CLOSE in response to every infraction.
(removing my comment as NAME posted what needs to be said couple of seconds earlier).
I'm going to re-close the can of worms I half-opened in my earlier comment and take this as editorial. From memory, the larger question of how to deal with junk is something we've discussed several times. If anyone wants to take that up, I'm happy to do that, but let's at least correct the obvious error.
It does make sense that we only allow 0x1c during the handshake.
NAME : Apologies, I misspoke -- I was speaking to interpretation of text, rather than the actual question itself.


Editorial NiTs when reviewing the invariant text. These NiTs address readability issues, and do not intend to change the meaning of the text.
Is there an associated issue?
Thanks Gorry, these are all quite helpful. 1) I understand the acronyms DCID and SCID, but these aren't really expanded on first use in sect 5.1. 2) There's mention of "the high bit", but there doesn't seem to be any statement of bit order saying. Is it worth declaring network byte/bit order earlier in section 4? This also clarifies the placement of fields within a byte, which was basically assumed knowledge otherwise. 3) Personally, I don't think: "The length of the Destination Connection ID is not specified" is a great choice of words. Surely, this needs to be specified somewhere for a protocol instance to use it? This reads better as something like: The length of the Destination Connection ID is not encoded in packets with a short header and is not constrained by this specification. 4) The brackets here seem to point to connection IDs in general, not the DCID "(see Section 5.3)". Is it worth writing that as "Connection IDs are described in Section 5", rather than use brackets? The same is true for versions. 5) In section 5.3: "the wrong endpoint" is taken as the wrong transport endpoint, but some of our IP and subIP colleagues would use the term endpoint for other uses, so one insertion of "transport" would perhaps help? 6) In the appendix A, this: "QUIC forbids acknowledgments of packets" - still has me wondering what is actually intended to be understood, because the list of bullets is not guarenteed ... : /The last packet before a long period of quiescence might be assumed to contain an acknowledgment (it should be assumed that QUIC could allow acknowledgments of packets that only contain ACK frames)/ I'm not sure what to say here I think the combination of not... forbids threw me off the meaning. Changes proposed in
Thanks for changing SCID/DCID to "Source Connection ID" and "Destination Connection ID", that seems like a good change.


Thanks to Lucas for finding the study. Note that I found at least one middlebox with a default timeout of 30s; that was not a home gateway included in this study.
says that "experience shows" you should send packets every 15-30 seconds, RFC4787 notwithstanding. Ideally, we would have a source that we could reference for this "experience."
Maybe see URL
I'm going to say that while the 15s floor is a good requirement, I don't want to cite RFC 8085 for this gem, which I think is straight-up misinformation: "NATs require a state timeout of 2 minutes or longer [RFC4787]." We do cite RFC 8085 elsewhere and that's fine, but the fact here is that the IETF has requested that NATs provide a state timeout of 2 minutes. That does not mean that they do.
I have seen anecdata as well that showing 30 seconds is an important threshold. I think the ref we now have is good.


This moves text from the section on 0-RTT to the section on continuing after Retry. Though 0-RTT is the only thing that is seriously affected in the current design, Initial packets might also be affected if we ever wanted to protect those with anything other than a static key.
says that after receiving a Retry, clients MUST NOT reset the packet number for any packet number space, and references 17.2.3 for more info. However, 17.2.3 only explains why this is the case for 0-RTT packets. We should either narrow the prohibition or expand the explanation.
Or maybe move the explanation that currently exists 17.2.3 to 17.2.5.3 and use it as an example (to avoid it being read as the only reason).
NAME does NAME editorial change address your issue?
Yes, this is fine.


I'd always heard this as "Authenticated Encryption with Additional Data," so it was -tls that sounded funny. But looking at 5116, it's actually "Associated" data and -tls is correct. Fixing in -transport.
I read this as referring to Authenticated at first.


This does a little shuffling to move some of the context-setting up. It then adds some more meat to the introductory section.
Thanks for the review Jana. I've taken most of this, with small tweaks only. Two things I didn't take: Interleaving explanation with references to sections sounded appealing, but it turns out to be both harder to read as the links break up the flow, and harder to scan for links (the structure section serves as an abbreviated and accessible ToC). I didn't move the connection piece below streams as I think that - at this level - talking about the connection is important context. I've moved the picture to the section, which didn't have that simple diagram (just some more complex ones), and I think benefits from it.
When I was reading quic-transport I found it very difficult to deal with the forward references. Streams are presented before connections, the overall state machine isn't introduced up front, packets come long after frames, and both are long before the wire formats are described. It might worth revisiting the section ordering.
At some point, transport was hugely restructured, and I believe many of those changes were good, but some didn't work out that well. I find streams coming before connections particularly odd.
The larger idea is "high-level concepts, then wire encoding" which I think makes sense. It puts the things that someone needs for a conceptual understanding of the protocol in one place, and the things that someone needs to implement the protocol elsewhere. Within the high-level concepts, I also find it strange that connections don't come before streams. However, the "connections" group also includes things that really don't deserve pride-of-place as the first couple things to be discussed, either.
I think that the main concept you need is that QUIC has a connection that is shared state between two endpoints. The later sections about connections go into more detail than you need to understand the stream sections. But I note that we don't properly provide that basic knowledge in our introductory parts, which are quite lean. Leanness is good, but we might have overdone it slightly in this regard.
Having worked my way through the spec, my suggested resolution to this is that Section 5.3 needs to be earlier, perhaps much earlier. At the least, it should open Section 5; some or all of it might belong in Section 1.
Thanks for working through this!


I see a couple of places where we require that values have high entropy. For instance of the packet header. The remainder of the first byte and an arbitrary number of bytes following it that are set to unpredictable values. The last 16 bytes of the datagram contain a Stateless Reset Token. and to ensure that it is easier to receive the packet than it is to guess the value correctly. In general, this means that things contain sufficient entropy (so, for instance, you might have a value with some fixed values and some high entropy values). In any case, I think the first of these should be "indistinguishable from random" and the second should be "containing at least 64 bits worth of entropy"
LGTM


This was never explicit for the client as it is usually not possible. in a way that perhaps was not anticipated.
is complete. Consequently, a server might expect 0-RTT packets to start with a However, this is not necessarily strictly true, because you might have the ACKs in 0.5RTT. Now you might say that the TLS stack reports the handshake as complete as soon as it receives Finished but that's an implementation detail
I think that this is true. The server can send before then, but we require that the client not install 1-RTT keys until it considers the handshake complete.
That's not entirely true. It's the server that needs to hold back keys (namely, the 1-RTT read key) until the handshake completes. The client can use keys as soon as they become available.
I think NAME is correct in sense that the following three events happen at the same moment on the client side: handshake is declared complete TLS stack provides the 1-RTT read and write keys TLS stack provides ClientFinished
Martin is correct. There are two different forms of handshake completion: complete when 1rtt can be sent and received, confirmed when the peer is known to be complete. The client cannot receive 0rtt acks before installing 1rtt keys, hence before hanshake is complete.
a nit, but LGTM


Those packets that are not authenticated can be discarded, but only if processing them has no permanent effect on existing state. This attempts to explain that.
S 5.2. says: Invalid packets without packet protection, such as Initial, Retry, or Version Negotiation, MAY be discarded. An endpoint MUST generate a connection error if it commits changes to state before discovering an error. I'm having trouble parsing this.
LGTM


This is a small tweak of Gorry's suggestion.
Looks good to me.
I promised to raise an issue relating to this text: "It can also improve connection throughput on severely asymmetric links; see Section 3 of [RFC3449]." This partly addresss the case, but it does not yet note the implications of the return path traffic on congestion of the return path. I think this is a useful addition, because it helps alert the reader of the tradeoff ... which can be quite important for half-duplex/shared radio, etc where the spectrum consumed sending an ACK can even outweigh the cost of sending a data packet (because they use different design of PHY). I suggest: "It can improve connection throughput using severely asymmetric links and can also reduce the volume of acknowledgment traffic using return path capacity; see Section 3 of [RFC3449]."


PTO packets always have to be ACK-eliciting, not just during the handshake, per existing text: "When a PTO timer expires, a sender MUST send at least one ack-eliciting packet in the packet number space as a probe,..." Also removed an "unless" clause, because if there's nothing to send, you don't arm the PTO timer.


I believe this covers the discussion on the issue. It also introduces some MUSTs which should've always been there. I don't have a strong opinion on the use of RFC 2119 language here, and I'm happy to remove all of that language if folks think that's better. I would like not to bikeshed this, but I would also like to be on a beach right now, enjoying a lovely drink and watching the sunset.
NAME wrote the following in his WGLC review (): Splitting this off from as a potential design issue.
This seems like a reasonable thing to do.
I'd support a can or a MAY, as an example when you might reset min_rtt, but I don't think we need a SHOULD here.
minrtt is currently only used for filtering unexpectedly large ackdelay's reported by the peer. If the path RTT truly inflates then the peer will be able to report much larger ackdelay values and get a much lower RTT sample accepted. But I don't think this is a security problem in practice. The only other consideration is that in future QUIC implementations may reuse this computed value for other purposes like delay based congestion control. It's hard to make a judgement call on how important it is to reset minrtt. I guess I'd be happy with a SHOULD but I can live with a MAY.
Jana made a good argument that persistent congestion could be declared due to an overly small RTT, so resetting minrtt could avoid an endpoint repeatedly declaring persistent congestion. That being said, the RTT samples also need to be too low to spuriously declare persistent congestion, so this is increasingly sounding like an attack scenario, rather than having a poor estimate of minrtt. I think a SHOULD is Ok now, but I'd like to state the best rationale we have for why it's a SHOULD and neither a MUST or a MAY.
I think there's one tricky point in resetting the minrtt after handshake confirmation. That is that all the packets can be acked with delays. IIRC, one of the reasons we were happy with using minrtt for capping latestrtt is that Initial and Handshake packets do not have ackdelays. If we are to suggest that an endpoint MAY reset minrtt when persistent congestion is declared, I think we should consider the risk of not having good estimate of minrtt, as we would have during the handshake.
I don't see that as a risk here. min_rtt doesn't consider any discretionary delays.
NAME That's exactly the problem. will be inflated by , unless and until the peer sends an immediate acknowledgement. And until that happens, will as well.
I think that's OK. In practice, immediate ACKs do happen, and even at low rates, this should resolve to a "true" minimum. And if it doesn't, then you have worse performance than you might had otherwise. You have to have a pretty poor network (or a terrible ACK generation regime) for minrtt not to eventually resolve to a useful minimum. Explaining this risk is fine, but the endpoint that does this made the choice to throw out minrtt and that's just a consequence of that.
On one hand I agree that we can expect to see immediate ACKs, especially when HTTP/3 is used. At the same time, I am not sure if resetting minrtt when observing persistent congestion is better than other cases where we might reset minrtt, but that we do not currently mention. Change of IPTTL could be one of the examples in this regard (IIRC NAME pointed out that we do not use that value for detecting path changes). To summarize, I think talking about resetting minrtt might be fine, though I am not sure if that discussion should be tied to persistent congestion.
To give a specific example, one problem that I know with the current definition of minrtt is that it cannot detect and correct RTT estimate of a congested path. The sole purpose of minrtt is to discard excessive ack-delays being reported by the receiver. This works by capping the lower bound of adjustedrtt to minrtt. The problem here is that minrtt is a value that is typically collected when the path is almost idle. Once the path becomes congested, the true RTT becomes much greater than minrtt. For instance, if RTT-on-idle is 10ms and if RTT-on-congestion is 50ms, minrtt would be 10ms, and therefore cannot be used to correct RTT estimates when the peer reports an excessive ackdelay of, say 25ms. The RTT estimate would become 25ms in such case. I tend to think that this might be a bigger problem of the original issue, and therefore that we should state that an endpoint can periodically re-collect minrtt estimates, if we are to make a change to the current definition of minrtt.
Based on agreement on the resolution in , which introduces normative language, let's mark this design.
Seems fine. I'm not 100% on the strength of the recommendation to reset after persistent congestion, but it's sensible advice.The resetting of min RTT is also specified with LEDBAT or BBR, which differ from each other and from this proposed spec. This is a complex issue, because resetting min RTT in LEDBAT "because of congestion" is a variation of the latecomer advantage. But as long as the text applies only to RENO as specified here, then I am fine with it.I like the cautionary text. Just one suggestion but LGTM regardless. Thank you for working on this.


This is an error that can be detected in frame parsing without additional context.
Over on URL we had a discussion about whether a MAXSTREAMS frame with value exceeding 2^60 generates a FRAMEENCODING or a STREAMLIMIT error. Draft 29 Section 19.11 says: corresponding type that can be opened over the lifetime of the connection. This value cannot exceed 2^60, as it is not possible to encode stream IDs larger than 2^62-1. Receipt of a frame that permits opening of a stream larger than this limit MUST be treated as a FRAMEENCODINGERROR. but as NAME points out, Section 4.5 says with a value greater than 2^60, this would allow a maximum stream ID that cannot be expressed as a variable-length integer; see Section 16. If either is received, the connection MUST be closed immediately with a connection error of type STREAMLIMITERROR Meanwhile, for the STREAMSBLOCKED frame in Section 19.15 we have: I suspect both Vidhi and I are correct, bogus MAXSTREAMS can be treated as either STREAMLIMIT or FRAME_ENCODING. If so, its probably a minor editorial fix to state either/or in both the sections.
NAME suggested that it should only be a FRAMEENCODINGERROR, as it's invalid to send a MAX_STREAMS larger than 2^60 under any circumstance, regardless of connection state, and I agree that's the clearest application of our principles.
I agree that requiring the use of FRAMEENOCDINGERROR is fine. Even though we occasionally allow an endpoint to use multiple error codes when there is more than one way of checking the status (i.e. FLOWCONTROLERROR vs. FRAMEENCODINGERROR for STREAM frames), the check required in this particular case is purely unrelated to the connection state. Hence my +1.
I would be happy with a single error too. I'm not fussy about the actual code used as long as the spec is clear and consistent. That means that we'd need to reduce STREAMS_BLOCKED to one error code too, and potentially any other situation that allows either.
I'm going to drop . I agree with the comments suggesting that FRAMEENCODINGERROR is the right answer. Note that you might still receive a different error, as we permit endpoints that detect multiple errors to send any that it detects, and this case clearly justifies use of STREAMLIMITERROR.


The original text here might be read to imply that a server can't send Retry if it receives an Initial that contains a token that came from NEW_TOKEN. That's wrong.


An update to I read the 4091 text as implying that probes were sent upon a loss.


I noticed this in the Transport Draft (ID-30): Is this intentional? Should an invalid transport parameter result in a frame encoding error?
I believe this aligns with our principles that a framing error which can be detected with no connection context is a FRAMEENCODINGERROR. Is there another error you'd suggest?
My concern is that transport parameter values have nothing to do with framing and using FRAMEENCODINGERROR is therefore misleading.
We only ever applied this principle to frames though, so I'm not sure we can use it as a precedent here. The name is indeed confusing, as it's FRAMEENCODINGERROR and not FRAMINGERROR or ENCODINGERROR.
I always rationalized this as "the encoded content of the CRYPTO frame is erroneous".
Well, if we've been through this before, I won't insist -- it's just something that jumped out at me.
That's even more confusing. The CRYPTO frame itself is totally fine. I'd be in favor of changing it to a TRANSPORTPARAMETERERROR.


In the HTTP/3 draft 30, section 4.4 on Server Push says and then adds in 10.4 (URL) I don't think normative requirements belong in security considerations except in reference to that requirement elsewhere. Also, requirements in general should clearly state who is responsible for implementing them, whereas the latter requirement implies a client (maybe). And sending CANCEL_PUSH is kind of like the MUST NOT, but not really the same thing; it seems a bit weak in response given that the server just tried to poison the client. I think the server MUST is an interop requirement because the client MUST NOT is a critical security requirement. I think both should be described in Server Push and merely noted in the Security Considerations, since they are really important. Way more important than the spec reads now. My guess is that this is editorial, but is the kind of thing that should be fixed before IESG last call.
The reason it's only a SHOULD CANCEL_PUSH is that the server can't know with certainty for which origins the client considers it authoritative. The server only knows for which origins it believes that it's authoritative. So having received a push the client considers out-of-bounds, the client can't know for sure whether it's a poisoning attempt or simply a difference in view. Yes, I think this is editorial, since the core request is to relocate existing text without changing the normative requirements.


Proposal to clarify "stream fragmentation". Suggested by Issue .
I don’t understand what section 21.6 is about… the title seems to suggests it is about IP Fragmentation and Reassembly Attacks, but elsewhere the spec already prohibits IP Fragmentation in section 14? And I have seen no encouragement to do any form of network-layer fragmentation - so I don’t understand a security consideration in this topic. Is this therefiore some other thing? It may about something different where a sender originates data with “holes” of missing packet to try and exercise the server? or soemthing? What is this?
Hi NAME 21.6 is titled "Stream Fragmentation and Reassembly Attacks". It refers to streams, not IP packets. The attack involves sending stream frames with holes such as and some receivers might allocate the entire stream receive buffer between 0 and 10000001. I found the section pretty clear, but we should make it clear to everyone - could you perhaps suggest text that would make it clearer?
OK. I see. I can try a PR, if I can find words that avoid "fragment": Is this close enough to start: /An adversarial sender might intentionally omit to send portions of the stream data causing the receiver to commit resources for the omitted data, this could cause a disproportionate receive buffer memory commitment and/or creation of a large and inefficient data structure.
Do we really need to avoid the term "fragment"? It's not the only overloaded networking term in the spec. Perhaps adding a "Note that stream fragmentation is unrelated to IP fragmentation" could suffice?
I don't think this clarification is necessary. The context makes it clear what fragmentation this is, and talking about IP fragmentation only leads to confuse the reader. (I like the first part of your PR though, as I've noted on my review of it.)


N was introduced in to describe how a sender might pace at a rate that allows it to send a congestion window's worth of data in a period that is shorter than an RTT. This PR clarifies text that goes into unnecessary detail, trying to describe assumptions (the network is spreading the packets out, for example, resulting in an ack clock).
Can we make progress in this one?
Yes, I'm waiting for the design issues to get done before moving on the remaining editorial ones.
I think that has happened now :-)
Nudge


Also removes odd MAY when suggesting that a sender can use TCP's pipeACK method. Addresses .
In Recovery-31 there are several references that I judge as normative and not informative ones. Section 7.8: "A sender MAY use the pipeACK method described in Section 4.3 of [RFC7661] to determine if the congestion window is sufficiently utilized." A normative reference necessary to follow if one selects to use the optional method.Sectiom 7.3.2: "Implementations MAY reduce the congestion window immediately upon entering a recovery period or use other mechanisms, such as Proportional Rate Reduction ([PRR]), to reduce the congestion window more gradually." Another case of a suggested mechanism, which if one is to follow it requires one to read that PPR reference. Secondly, why the alternative reference style.Section 8.3: "Markings can be treated as equivalent to loss ([RFC3168]), but other responses can be specified, such as ([RFC8511]) or ([RFC8311])." So RFC3168 is the normative specification for the treat it equal to loss. RFC8511 is the TCP alternative back-off an experimental specification for the alternative. And RFC8311 is the process for performing other experiments with alternative responses. I judge all necessary depending on what one intendeds to do. So to my understanding this results in two downrefs to experimental specifications. I think that is acceptable in the context they are used in this document. But to be tried in IETF last call. However, I think I don't want to generally recommend them to be acceptable as downrefs so I have no intentions to promote them for the Downref registry.
Moving discussion from the PR to the issue. NAME : I'm not the expert on what should be normative, but these were intentionally marked as informative references since they're all TCP specific RFCs. The pipeACK reference in particular seems like an odd one. If this is what the chairs want, then I'm ok with it as an editor, but these choices were intentional. NAME : I see what NAME is saying, but I also see some wiggle room in RFC 3967, Section 1.1: When a reference to an example is made, such a reference need not be normative. For example, text such as "an algorithm such as the one specified in [RFCxxxx] would be acceptable" indicates an informative reference, since that cited algorithm is just one of several possible algorithms that could be used. I am also somewhat concerned that trying to get the downrefs into the registry is more likely to run into trouble (Sec 3) This procedure should not be used if the proper step is to move the document to which the reference is being made into the appropriate category. It is not intended as an easy way out of normal process. Rather, the procedure is intended for dealing with specific cases where putting particular documents into the required category is problematic and unlikely ever to happen.
Thanks for digging up the relevant text, NAME Based on that text from RFC 3967, it doesn't seem like we need to make these references normative. I agree that the pipeACK reference is odd, but I think that's probably because of the use of MAY there. I'm happy to replace the MAY with a can -- it is meant to present an option, not a permission -- and make that reference informative as well. NAME : thoughts?
I think at last RFC3168 should be referenced normatively somewhere (probably section 7.1 though) because you need to know what the ECN bits are in order to do anything with it. I guess because of section 7.1. RFC8311 should also be a normative reference.
NAME RFC3168 is already a normative reference in the transport doc, which in my view is appropriate.
Was wondering about this as well given the transport doc is of course a normative reference of this doc. However, as this doc uses terms like CE also directly itself I think the right practice is to also have it as a normative reference here, and as you have to read it no matter what, it definitely shouldn't hurt.
talks about ECN-CE markings, but really all of Section 7 talks about a response to an increase in ECN-CE count by the peer, which is defined in the transport doc. This does point to a small editorial issue. The text in Section 4.1 says: This really should be:
Good point on the editorial issue.
With that change, the only thing we're left with is . This does require an understanding of ECN, and it has RFC 2119 keywords, but it's really an optional mechanism that is suggested. We could remove the RFC 2119 words there, since they're really unnecessary.
My point is that RFC3168 should be normative because not matter what you need to understand what an ECN-CE mark is and that is defined/specified in RFC3168.
Mirja has a point, see URL
-transport is a normative reference. So 3168 is already (transitively) a normative reference. The question is whether this specific text requires anything from 3168 to understand. I don't see that it does, especially given the suggested change, which talks about a concept defined in -transport, not 3168.
has made these editorial changes; NAME can you see if anything else needs to be done here?
I will note that it was never my intention to let any of theses Experimental or informational TCP specs to be added into the downref registry by this last call. So, the pipeack reference needs to be reformulated to not be a normative one. As currently stated it is a clear normative reference per IESG Statement ([URL]) , and especially "Note 1: Even references that are relevant only for optional features must be classified as normative if they meet the above conditions for normative references. " You need to find a formulation that normative describes what the implementation MAY do, and then you can exemplify with the TCP version. Then one in Section 7.3.2 I can accept as still being informative as the MAY statement is related to the primary aspect of the sentence and the reference to PPR is exemplifying. Still this is very much on the border and may still generate a comment in IESG. For RFC 3168 I don't understand why not making it to a normative one. There are no issues with its status as it is proposed standard. And simply the security consideration section still implies that you need to know what a ECN-CE mark is to interpret this. So please make this one normative.
I'm ok with RFC 3168 being normative, particularly since it's already normative in transport, but would like to avoid making the others normative.
NAME if pipeack is not normative then a reformulation is necessary of that sentence in Section 7.8 to not state is as a mechanism that MAY be implemented.
The sentence currently has a can, not a MAY: "A sender can use the pipeACK method described in Section 4.3 of {{?RFC7661}} to determine if the congestion window is sufficiently utilized." But I sent out PR to make it clearer that pipeACK is an example, PTAL and see if that helps.
NAME what PR proposes removing that first sentence do resolve the pipeack to me. So merging , and moving RFC3168 to normative will resolve this issue to my understanding.
Thanks! I'll update PR to include moving RFC3168 to normative.
NAME NAME : I believe this is purely editorial, but I'd like to confirm.
We've meandered to a route that AFAICT makes reference types accurate to their existing intent, or avoids them becoming normative. I'm happy for this to be editorials.
I think that this is the most appropriate response.