Rate and ceil are controlled with 2 bucket: one with tokens for the rate and one with ctokens for the ceil. If a class sends a packet, a token/ctoken is used. But new tokens/ctokens are arriving at the speed of the rate/ceil. The size of the bucket is the configured burst/cburst. There is also a minimal burst needed so you can send something. And this is related to the maximum packet size (1500 byte) and the timer resolution.
If a child is using a token to send a packet, the same tocken is requested from the parent. So the child class is using the tokens/ctokens of it's parent. And without tokens, the parent can't give remaining bandwidth to it's child classes.
It sounds stupid, but burst is made for burstable traffic like http and not for big bursts. So the tokens/ctokens that are used when you burst traffic, can be rebuild when there is less traffic.
If you configure a htb class, you can overrule the default cburst and burst. But what happens if you add these parameter to parent and/or child classes?
I added an extra burst to 1 child class
Used script (download)
tc qdisc del dev lo root tc qdisc add dev lo root handle 1: htb TC="tc class add dev lo parent" $TC 1: classid 1:1 htb rate 300kbps $TC 1:1 classid 1:10 htb rate 150kbps ceil 300kbps burst 500kb $TC 1:1 classid 1:11 htb rate 150kbps ceil 300kbps
Class 1:10 has an extra burst. So the size of the bucket of class 1:10 is not zero, but it holds some tokens. That's why class 1:10 is sending more then it's configured rate, it uses it's own tokens. At point 5.5 all tokens of class 1:10 is used so the bandwidth becomes it's configured rate.
At the same time, you can see an other intersting behaviour. The parent class is sending more then it's ceil. That's because each child class (1:10 1:11) can always send the configured rates AND class 1:10 can even send more because it has a burst defined. But if the parent class is sending more than it's ceil, it's ctokens is going negative. In the next test, I'm trying to find out what this means
What happens if the ctokens of the parent is negative? So after some time, I stop class 1:10 from sending more data.
Used script (download)
tc qdisc del dev lo root tc qdisc add dev lo root handle 1: htb TC="tc class add dev lo parent" $TC 1: classid 1:1 htb rate 300kbps $TC 1:1 classid 1:10 htb rate 150kbps ceil 300kbps burst 500kb $TC 1:1 classid 1:11 htb rate 150kbps ceil 300kbps
The first part of the graph is the same as in test 1. But after 12 sec, class 1:10 stops sending data. At that point, only class 1:11 is sending data. And it's sending only 150k/s. The parent class (1:1) has still a negative ctoken bucket so it can't give the remaining 150k/s to class 1:11. But when the ctoken bucket can refill because class 1:1 is sending less then it's rate. If the ctoken bucket is filled again, the parent class can give the remaining 150K/s to class 1:11.
At point 14.5, the ctoken bucket of class 1:1 is zero again and after that point, class 1:11 can borrow the remaining bandwidth from the parent. It's not totally clear because the blue and green line are the same.
What can we do aginst the "dip" at 12s? I tried to add a burst and/or a cburst to the parent class.
tc qdisc del dev lo root tc qdisc add dev lo root handle 1: htb default 10 TC="tc class add dev lo parent" $TC 1: classid 1:1 htb rate 300kbps burst 500kb $TC 1:1 classid 1:10 htb rate 150kbps ceil 300kbps burst 500kb $TC 1:1 classid 1:11 htb rate 150kbps ceil 300kbps
tc qdisc del dev lo root tc qdisc add dev lo root handle 1: htb default 10 TC="tc class add dev lo parent" $TC 1: classid 1:1 htb rate 300kbps burst 500kb cburst 500kb $TC 1:1 classid 1:10 htb rate 150kbps ceil 300kbps burst 500kb $TC 1:1 classid 1:11 htb rate 150kbps ceil 300kbps
Test 5 : parent burst 500kb cburst 500kb
tc qdisc del dev lo root tc qdisc add dev lo root handle 1: htb default 10 TC="tc class add dev lo parent" $TC 1: classid 1:1 htb rate 300kbps burst 500kb cburst 500kb $TC 1:1 classid 1:10 htb rate 150kbps ceil 300kbps burst 500kb $TC 1:1 classid 1:11 htb rate 150kbps ceil 300kbps
So there is no way to prevent to "dip" caused by the burst of cllass 1:10.
Notice that it in the last graph, the parent allows a burst in it's ceil. So as long as the parent has ctokens, both classes can use 300kbps, the configured ceil. But at 4s the ctokens of the parent are all used so class 11 dropes back to it's rate, but class 10 can use it's own tokens to get 300kbps.
What if I give the parent some burst and cburst? And no burst/cburst to the child classes?
tc qdisc del dev lo root tc qdisc add dev lo root handle 1: htb default 10 TC="tc class add dev lo parent" $TC 1: classid 1:1 htb rate 300kbps burst 500kb $TC 1:1 classid 1:10 htb rate 150kbps ceil 300kbps $TC 1:1 classid 1:11 htb rate 150kbps ceil 300kbps
tc qdisc del dev lo root tc qdisc add dev lo root handle 1: htb default 10 TC="tc class add dev lo parent" $TC 1: classid 1:1 htb rate 300kbps cburst 500kb $TC 1:1 classid 1:10 htb rate 150kbps ceil 300kbps $TC 1:1 classid 1:11 htb rate 150kbps ceil 300kbps
Test 8 : parent burst 500kb cburst 500kb
tc qdisc del dev lo root tc qdisc add dev lo root handle 1: htb default 10 TC="tc class add dev lo parent" $TC 1: classid 1:1 htb rate 300kbps burst 500kb cburst 500kb $TC 1:1 classid 1:10 htb rate 150kbps ceil 300kbps $TC 1:1 classid 1:11 htb rate 150kbps ceil 300kbps
If the parent class has a burst and cburst, but these tokens can only be used if the parent has tokens AND ctokens. These extra tokens are used to give each class all the bandwidth it wants. But the rate is still limited by the ceil. When all tokens and ctokens of the parent are used, the rate of the parent droppes back to it's rate.
After 12 seconds, class 10 stops sending. And after that parent 1:1 can rebuild it's supply of tokens and ctokens.
In the next serie of tests, I give the 2 child classes a lower rate, 100kbps. So the parent has always some remaining bandwidth to give away.
Used script (download)
tc qdisc del dev lo root tc qdisc add dev lo root handle 1: htb default 10 TC="tc class add dev lo parent" $TC 1: classid 1:1 htb rate 300kbps $TC 1:1 classid 1:10 htb rate 100kbps ceil 300kbps burst 1000kb cburst 1000kb $TC 1:1 classid 1:11 htb rate 100kbps ceil 300kbps
Class 11 can send it's rate but class 10 can send more. It has tokens in it's bucket to send up to it's rate. So the parent is sending more then it's rate and it's token/ctoken bucket is negative.
At point 7.5 class 10 has no tokens left, so the rate of class 10 droppes to it's rate 100kbps. After that point, the parent can refill it's tokens/ctokens bucket. At point 12, the parent can give it's remaining bandwidth away, so each class get's 100kbps.