Twitter and the JVM

Oracle, November 5, 2012

Use

The JVM is used throughout

• Frontend web server
• Application services
• Data services
• Ads backends
• (New) web frontends
• ...

Systems are both in Scala and Java; most are using ParNew+CMS.

JVM development

We have a VM team (Brandon, Kaushik) contributing to OpenJDK.

• G1 contributions (to get our commit bit); Cassandra using G1
• Profiling, debugging tools: light-weight full-stack CPU profiling

Concerns

Garbage collection, garbage collection, garbage collection, garbage collection, garbage collection, garbage collection, garbage collection, garbage collecton, garbage collection, garbage collection, garbage collection, garbage collection, garbage collection, garbage collection, garbage collection, garbage collection, garbage collection, garbage collection, garbage collecton, garbage collection, garbage collection, garbage collection, garbage collection, garbage collection, garbage collection, garbage collection, garbage collection, garbage collection, garbage collection, garbage collecton, garbage collection, garbage collection, garbage collection, garbage collection, garbage collection, garbage collection, garbage collection, garbage collection, garbage collection, garbage collection, garbage collection, garbage collection, garbage collection, garbage collecton, garbage collection, garbage collection, garbage collection, garbage collection

Concerns (part 2)

Concerns (really)

We have good throughput (GC overhead is usually .2-.5%)

But tail latencies are very important. Effect is compounded in distributed systems.

Most of our systems are stateless.

Tooling

Google-perftools compatible tooling

Heap profilers

github.com/mariusaeriksen/heapster

github.com/twitter/jvmgcprof

CPU, contention prolfilers

github.com/twitter/util/tree/master/util-jvm

Accessible via HTTP on production systems.

Systemic GC robustness

Still experimental:

RPC: building in GC avoidance

Speculative execution.

Speculative future work

To minimize tail latency effects, we run with the largest heaps possible.

But we want to share, to colocate (see: Mesos). Our cluster scheduler should be able to make latency-heap-size tradeoffs (eg. to do better binpacking or to oversubscribe).

• Backpressure / dynamically changing heap size ( -Xmx )

Thank you