Part 5: Understanding Amazon ElastiCache Internals: Memory Allocation and Eviction

Amazon ElastiCache internally uses Memcached 1.4.5 engine. We are going to explore the memory allocation and eviction policies of Amazon ElastiCache through our understanding of memcached internals in this article. If you are a complex and extensive user of Amazon ElastiCache it is better to have detailed understanding of how the internals work, to avoid leakages and maintain overall optimum performance.

How is it organized internally?

Amazon ElastiCache node usually breaks the allocated memory into smaller parts called pages. Each page is usually 1 megabyte in size. Each page is then assigned to a slab class when necessary. Each slab class is in turn divided into chunks of a specific size. The chunks in each slab have the same size.

Following diagram illustrates the above divisions:

If you are using memcached on EC2 you can view the slab, chunk and byte details by using the following command $. /memcached -vv

$ ./memcached -vv
slab class 1: chunk size 80 perslab 13107
slab class 2: chunk size 104 perslab 10082
slab class 3: chunk size 136 perslab 7710
slab class 4: chunk size 176 perslab 5957
slab class 5: chunk size 224 perslab 4681
slab class 6: chunk size 280 perslab 3744
slab class 7: chunk size 352 perslab 2978
slab class 8: chunk size 440 perslab 2383
slab class 9: chunk size 552 perslab 1899
slab class 10: chunk size 696 perslab 1506
[...etc...]

From the output of the above command you can observe that there is page with 80 byte chunks (slab class 1), a page with 104 byte chunks (slab class 2). In slab class 1, each chunk is 80 bytes and each page can then contain 13,107 chunks (1 MB / 80 bytes). This continues all the way up the slab chain till 1 megabyte (with growth factor of ~1.25). Note: There can be multiple pages assigned for each slab-class, but as soon as a page is assigned to a slab-class, it is permanent.

When you are storing items in Amazon ElastiCache, they are pushed into the slab class of the nearest fit. If your key + miscellaneous data + value is 70 bytes total, it will go into slab class 1, with an overhead loss of 10 bytes per item. If your data is 90 bytes total, it will go into Slab class 2, with an overhead of 14 bytes. If your cache access pattern ends up putting 90% of your pages in slab class 2, there will be less memory available for slab class 3. This simple model is followed by memcached engine to avoid memory defragmentation and get better speed in performance versus memory usage tradeoff.

When a page is full (meaning all chunks in the page are filled) and we still need to add another item, engine will fetch a new free page from the pool, assign it to the specified slab-class (depending upon item size), partitions it into chunks and gets the first available chunk to store the new item. Note: These 1024 byte pages are assigned on a FCFS basis (first come-first served), to the slab classes.

When there are no more pages left to assign to slab class, it will use LRU algorithm to evict items to reclaim memory. In the following section you can see in detail about the eviction mechanism of Amazon ElastiCache works.

When items are evicted?

Memory for an item is not actively reclaimed in Amazon ElastiCache. The memcached engine does not have in-built background threads that explicitly expires item and reclaims chunks. For instance, if you store an item and it expires, it still sits in the LRU cache at its position, until it falls to the end of the cache and is reused. However, if you fetch an expired item, memcached will find that the item is expired and free its memory for reuse.

To explain eviction mechanism in detail; Items are evicted from Amazon ElastiCache if they are expired or the slab class is completely out of free chunks and there are no free pages to assign to a slab class. In case there are no free chunks, or no free pages in the appropriate slab class, Amazon ElastiCache will look at the LRU in tail to reclaim an item. Basically, it will search the last few items in the “end” and identifies the ones that are already expired, makes it free for reuse. If it cannot find an expired item on the end, it will "evict" one which has not yet expired. Actually you could end up with one slab class constantly evicting recently used items, on the other hand another slab having a bunch of old items that just sit around. For example: When we need a 104 byte chunk, it will evict a 104 byte chunk, even though there might be a 280 byte chunk that is even older. This explains the internal workings that “Each slab-class has its own LRU and statistical counter updates, it behaves like a separate cache itself, it is not global LRU, but slab class LRU in short”.

How to reduce memory overheads?

If you are running Amazon ElastiCache clusters of few MB/GB overheads will not matter much. But if you are running Amazon ElastiCache clusters spanning in Hundreds of GB- > TB, you will end up losing lots of allocated memory as overheads if the chunk size and growth factor is not planned properly.

Since memcached engine of Amazon ElastiCache does defrag, you need to plan for better compaction and avoid overheads to keep the chunk sizes closer to your item sizes. This is not a one-time activity, but a periodical activity aligned to your cache growth.

A simple approach that can be followed is:

· Identify the item size usage and cache access patterns periodically

· Set the initial chunk size and growth factors accordingly in AWS console or API's for Amazon ElastiCache. You can also set the factor using (-f) and the initial chunk-size (-s) options if its memcached on EC2.

· If item sizes are big and predictable it is recommended to have bigger chunks and growth factors, if the item sizes are varied it is better to have smaller initial size and growth factor. This will keep the wastage minimal.

Disclaimer: The article is entirely based on the premise that Amazon ElastiCache uses memcached 1.4.5 as the engine. If AWS team has customized some sections of memcached engine, the author will not be liable for misinforming readers.

http://code.google.com/p/memcached/wiki/NewUserInternals

http://www.adayinthelifeof.nl/2011/02/06/memcache-internals/

Alifitle

Part 5: Understanding Amazon ElastiCache Internals: Memory Allocation and Eviction

No comments:

Featured post

Billion Messages - Art of Architecting scalable ElastiCache Redis tier

Tags