[RFC/PATCH] zcache/ramster rewrite and promotion

[Dan Magenheimer]

> From: Pekka Enberg [mailto:penberg at kernel.org]
> Sent: Tuesday, July 31, 2012 2:54 PM
> 
> On Tue, Jul 31, 2012 at 11:18 PM, Dan Magenheimer
> <dan.magenheimer at oracle.com> wrote:
> > diffstat vs 3.5:
> >  drivers/staging/ramster/Kconfig       |    2
> >  drivers/staging/ramster/Makefile      |    2
> >  drivers/staging/zcache/Kconfig        |    2
> >  drivers/staging/zcache/Makefile       |    2
> >  mm/Kconfig                            |    2
> >  mm/Makefile                           |    4
> >  mm/tmem/Kconfig                       |   33
> >  mm/tmem/Makefile                      |    5
> >  mm/tmem/tmem.c                        |  894 +++++++++++++
> >  mm/tmem/tmem.h                        |  259 +++
> >  mm/tmem/zbud.c                        | 1060 +++++++++++++++
> >  mm/tmem/zbud.h                        |   33
> >  mm/tmem/zcache-main.c                 | 1686 +++++++++++++++++++++++++
> >  mm/tmem/zcache.h                      |   53
> >  mm/tmem/ramster.h                     |   59
> >  mm/tmem/ramster/heartbeat.c           |  462 ++++++
> >  mm/tmem/ramster/heartbeat.h           |   87 +
> >  mm/tmem/ramster/masklog.c             |  155 ++
> >  mm/tmem/ramster/masklog.h             |  220 +++
> >  mm/tmem/ramster/nodemanager.c         |  995 +++++++++++++++
> >  mm/tmem/ramster/nodemanager.h         |   88 +
> >  mm/tmem/ramster/r2net.c               |  414 ++++++
> >  mm/tmem/ramster/ramster.c             |  985 ++++++++++++++
> >  mm/tmem/ramster/ramster.h             |  161 ++
> >  mm/tmem/ramster/ramster_nodemanager.h |   39
> >  mm/tmem/ramster/tcp.c                 | 2253 ++++++++++++++++++++++++++++++++++
> >  mm/tmem/ramster/tcp.h                 |  159 ++
> >  mm/tmem/ramster/tcp_internal.h        |  248 +++
> > 28 files changed, 10358 insertions(+), 4 deletions(-)
> 
> So it's basically this commit, right?
> 
> https://oss.oracle.com/git/djm/tmem.git/?p=djm/tmem.git;a=commitdiff;h=22844fe3f52d912247212408294be33
> 0a867937c
> 
> Why on earth would you want to move that under the mm directory?

Hi Pekka --

Thanks for your reply and question.

MM means "memory management" and zcache manages physical memory
to allow more pages of data to be stored in RAM.  So it seems a
logical place.  It's not a block driver, or a network driver,
or a device driver, or a filesystem... do you have a different
location in the kernel in mind?

Zcache does it a bit differently than all the other parts of mm
because it needs to; because all the other parts of mm try to
maximize the amount of physical memory that is directly addressable
by threads but one can't directly address pages that have been compressed.
So zcache uses the transcendent memory approach (via cleancache
and frontswap) to compress/decompress clean pagecache pages and
swap pages "on demand".  The tmem design also nicely handles
both the fact that the degree of compression is unpredictable and
the fact that the fraction of fixed total RAM used for compressed
pages vs "normal uncompressed mm" pages needs to be very dynamic.

Ramster does the same thing but manages it peer-to-peer across
multiple systems using kernel sockets.  One could argue that
the dependency on sockets makes it more of a driver than "mm"
but ramster is "memory management" too, just a bit more exotic.

Thanks,
Dan