Re: Vector vs std::vector

On 01/29/2014 07:08 AM, Kinkie wrote:

> Amos has asked me over IRC to investigate any performance
> differences between Vector and std::vector. To do that, I've
> implemented astd::vector-based implementation of Vector
> (feature-branch: lp:~squid/squid/vector-to-stdvector).

Does Launchpad offer a way of generating a merge patch/diff on the site?
Currently, I have to checkout the branch and do "bzr send" to get the
right diff. Is there a better way?

> I've then done the performance testing using ab. The results are in: a
> Vector-based squid is about 3% speedier than a std::vector based
> squid.

> This may also be due to some egregious layering by users of Vector. I
> have seen things which I would like to correct, also with the
> objective of having Vector implement the same exact API as std::vector
> to make future porting easier.

Can you give any specific examples of the code change that you would
attribute to a loss of performance when using std::vector? I did not
notice any obvious cases, but I did not look closely.

> test conditions:
> - done on rs-ubuntu-saucy-perf (4-core VM, 4 Gb RAM)
> - testing with ab. 1m requests @10 parallelism with keepalive,
> stressing the TCP_MEM_HIT code path on a cold cache
> - test on a multicore VM; default out-of-the-box configuration, ab
> running on same hardware over the loopback interface.
> - immediately after ab exits, collect counters (mgr:counters)
>
> numbers (for trunk / stdvector)
> - mean response time: 1.032/1.060ms
> - req/sec: 9685/9430
> - cpu_time: 102.878167/106.013725

I hate to be the one asking this, but with so many red flags in the
testing methodology, are you reasonably sure that the 0.28 millisecond
difference does not include 0.29+ milliseconds of noise? At the very
least, do you consistently get the same set of numbers when repeating
the two tests in random order?

BTW, your req/sec line is redundant. In your best-effort test, the proxy
response time determines the offered request rate:

   9685/9430 = 1.027 (your "3% speedier")
  1.060/1.032 = 1.027 (your "3% speedier")

  10*(1000/1.032) = 9690 (10-robot request rate from response time)
  10*(1000/1.060) = 9434 (10-robot request rate from response time)

Cheers,

Alex.
Received on Wed Jan 29 2014 - 18:53:00 MST