The ongoing discussion about the Reiser4 filesystem [story] continues on the lkml. Jeff Garzik discussed the complexity introduced by a plugin layer [story], suggesting it is really a second VFS, "furthermore, it completely changes the notion of what a Linux filesystem is. Currently, each Linux filesystem is a tightly constrained set of metadata support. reiser4 changes 'tightly constrained' to 'infinity'. While that freedom is certainly liberating, it also has obvious support costs due to new admin paradigms and customer configuration possibilities."

Linux creator Linus Torvalds weighed in on the discussion, "as long you call them 'plugins' and treat them as such, I (and I suspect a lot of other people) are totally uninterested, and in fact, a lot of people will suspect that the primary aim is to either subvert the kernel copyright rules, or at best to create a mess of incompatible semantics with no sane overlying rules for locking etc." He went on to add, "as far as I'm concerned, the problem with reiser4 is that it hasn't tried to work with the VFS people. Now, I realize that the main VFS people aren't always easy to work with (Al and Christoph, take a bow), but that doesn't really change the basic facts. Al in particular is _always_ right. I don't think I've ever had the cojones to argue with Al.."

Later in the same thread, Andrew Morton [interview] noted that he's currently reviewing the code, "meanwhile here's poor old me trying to find another four hours to finish reviewing the thing." Regarding the code he added, "the writeout code is ugly, although that's largely due to a mismatch between what reiser4 wants to do and what the VFS/MM expects it to do. If it works, we can live with it, although perhaps the VFS could be made smarter." He then suggested, "I'd say that resier4's major problem is the lack of xattrs, acls and direct-io. That's likely to significantly limit its vendor uptake." As for the plugin debate, Andrew said, "the plugins appear to be wildly misnamed - they're just an internal abstraction layer which permits later feature additions to be added in a clean and safe manner. Certainly not worth all this fuss."

From: Jeff Garzik [email blocked]
To: Pavel Machek [email blocked]
Subject: Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion
Date:	Thu, 27 Jul 2006 13:56:17 -0400

Pavel Machek wrote:
> Hi!
> 
>>> of the story for me. There's nothing wrong about focusing on newer code,
>>> but the old code needs to be cared for, too, to fix remaining issues
>>> such as the "can only have N files with the same hash value". 
>>>
>> Requires a disk format change, in a filesystem without plugins, to fix it.
> 
> Well, too bad, if reiser3 is so broken it needs on-disk-format-change,
> then I guess doing that change is the right thing to do...

Actually, there is reiser4 brokenness lurking in Hans' statement, too:

A filesystem WITH plugins must still handle the standard Linux 
compatibility stuff that other filesystems handle.

Plugins --do not-- mean that you can just change the filesystem format 
willy-nilly, with zero impact.

	Jeff



From: Hans Reiser [email blocked]
Subject: Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion
Date:	Thu, 27 Jul 2006 20:26:12 -0600

Jeff Garzik wrote:

>
> Actually, there is reiser4 brokenness lurking in Hans' statement, too:

Where!  Someone tell me!;-)

>
> A filesystem WITH plugins must still handle the standard Linux
> compatibility stuff that other filesystems handle.

Hmm, you mean we should first implement regular unix file plugins before
implementing enhanced functionality ones?  Are you aware that reiser4
plugins are per file, and thus if a user selects a plugin that is not
the default, and which has user visible semantic differences, it means
they said they want non-standard behavior?

>
> Plugins --do not-- mean that you can just change the filesystem format
> willy-nilly, with zero impact.

Yes they do.....

>
>     Jeff
>



From: Jeff Garzik [email blocked]
Subject: metadata plugins (was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion)
Date:	Fri, 28 Jul 2006 07:15:11 -0400

Hans Reiser wrote:
> Jeff Garzik wrote:
>> Plugins --do not-- mean that you can just change the filesystem format
>> willy-nilly, with zero impact.

> Yes they do.....

Take off your marketing cap for a second :)

Plugins do not eliminate the going-back-to-older-kernels issue, as 
discussed at length in the recent ext4 thread.  Plugins complicate, 
rather than simplify, the issue of determining the compatibility between 
the kernel's metadata handling code and on-disk metadata.

Quite simply, there is always a compatibility impact when the filesystem 
format changes.  Enterprise customers will inevitably run several 
-generations- of your software, and they are very aware of migration 
costs when dealing with hardware+software life cycles in the 5-10 year 
range.  Power users and kernel developers bounce between kernel versions 
all the time.

This is the same problem I described in the ext4 thread, then summarized 
as "what ext3 am I talking to, today?"  You can easily rewrite that as 
"what reiser4 set of plugins am I talking to, today?"

Plugins == a ton of new software to be versioned and tracked.  Rather 
than just track "reiser4", you must now track the collection of plugin 
versions as well.  OBVIOUS increased complexity, and OBVIOUS additional 
admin learning curve for the serious IT shop.


Now prepare for the bombshell revelation...


THAT SAID, I do think fs {algorithm|metadata} plugins are a very 
interesting idea.

The plugin infrastructure is the logical result of trying to support 
multiple incompatible {inode, dir, file data, ...} object types in the 
same Linux filesystem.  Looking at in-tree Linux filesystems, one can 
see common design patterns in filesystems that hand-code support for 
multiple metadata format versions -- Al Viro's fs/sysv hacks being a 
simple and elegant example.

It is then simple to follow that train of logic:  why not make it easy 
to replace the directory algorithm [and associated metadata]?  or the 
file data space management algorithms?  or even the inode handling?  why 
not allow customers to replace a stock algorithm with an exotic, 
site-specific one?

In essence, reiser4 is not really a filesystem, but a second VFS, with a 
few stock metadata modules.

Furthermore, it completely changes the notion of what a Linux filesystem 
is.  Currently, each Linux filesystem is a tightly constrained set of 
metadata support.  reiser4 changes "tightly constrained" to "infinity". 
  While that freedom is certainly liberating, it also has obvious 
support costs due to new admin paradigms and customer configuration 
possibilities.

I don't want to be the distro support person trying to fix a crash in 
"reiser4", where the customer has secretly replaced the standard inode 
data structure with a plugin written by an intern, and secretly replaced 
the directory algorithm with a closed source plugin from PickYourVendor. 
  Trying picking through that mess with a filesystem debugger.

The reiser4 plugin system is far better implemented as the VFS level, as 
an optional library, like fs/libfs.c.  Such a library would contain 
factored-out infrastructure common to all filesystems that support 
multiple versions of the same object type, thereby shrinking the "real" 
filesystem code.  Then add a tiny bit of code that allows addition and 
removal of object types within each filesystem.

At that point, you have reduced a Linux filesystem to binding between a 
common name ("reiser4", "ext3") and a set of cooperating modules.

A very interesting train of thought.  But not one that belongs in the 
kernel in its current form.  If you hide a second VFS inside fs/reiser4, 
I GUARANTEE that you will get the locking and multi-threading wrong.  I 
guarantee your code will be under-reviewed, and NAK'd.  You lose all the 
testing that would be gained by others travelling your code paths, if 
the code was implemented generically in fs/libplugin.c rather than 
fs/reiser4/*

To move forward towards the goal of merging reiser4 into the kernel, you 
must recognize two distinctly SEPARATE ISSUES, and deal with them 
separately:

1) merging the latest whiz-bang collection of filesystem algorithms, 
under the filesystem name "reiser4"
2) adding the plugin concept to the Linux VFS

I don't know if #2 will pass consensus, but I certainly think we should 
at least experiment in that direction.  Your guys definitely have a lot 
of good ideas, and I'm glad Linux is seeing a fresh injection of new ideas.

I just don't think its a good idea to merge a filesystem called "foo" 
that can be completely redefined in the field.  Or least, not without 
thinking a lot more on the impact.  Its a VFS _and_ a filesystem, and 
should be evaluated as such.

	Jeff



From: "Horst H. von Brand" [email blocked]
Subject: Re: metadata plugins (was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion) 
Date:	Fri, 28 Jul 2006 10:02:04 -0400

Jeff Garzik [email blocked] wrote:

[...]

> It is then simple to follow that train of logic:  why not make it easy
> to replace the directory algorithm [and associated metadata]?  or the
> file data space management algorithms?  or even the inode handling?
> why not allow customers to replace a stock algorithm with an exotic,
> site-specific one?

IMVHO, such experiments should/must be done in userspace. And AFAICS, they
can today.

Go wild, but leave the kernel alone.
-- 
Dr. Horst H. von Brand                   User #22616 counter.li.org
Departamento de Informatica                     Fono: +56 32 654431
Universidad Tecnica Federico Santa Maria              +56 32 654239
Casilla 110-V, Valparaiso, Chile                Fax:  +56 32 797513



From: David Masover [email blocked]
Subject: Re: metadata plugins (was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion)
Date:	Fri, 28 Jul 2006 10:48:34 -0500

Horst H. von Brand wrote:
> Jeff Garzik [email blocked] wrote:
> 
> [...]
> 
>> It is then simple to follow that train of logic:  why not make it easy
>> to replace the directory algorithm [and associated metadata]?  or the
>> file data space management algorithms?  or even the inode handling?
>> why not allow customers to replace a stock algorithm with an exotic,
>> site-specific one?
> 
> IMVHO, such experiments should/must be done in userspace. And AFAICS, they
> can today.

inode handling?  Really?

But what's wrong with people doing such experiments outside the kernel? 
  AFAICS, "exotic, site-specific one" is not something that would be 
considered for inclusion.



From: Linus Torvalds [email blocked]
Subject: Re: metadata plugins (was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion)
Date:	Fri, 28 Jul 2006 09:33:56 -0700 (PDT)



On Fri, 28 Jul 2006, David Masover wrote:
> 
> But what's wrong with people doing such experiments outside the kernel?
> AFAICS, "exotic, site-specific one" is not something that would be considered
> for inclusion.

Here's a few ground rules at least from my viewpoint:

 - as long you call them "plugins" and treat them as such, I (and I 
   suspect a lot of other people) are totally uninterested, and in fact, a 
   lot of people will suspect that the primary aim is to either subvert 
   the kernel copyright rules, or at best to create a mess of incompatible 
   semantics with no sane overlying rules for locking etc.

 - the kernel does not, and _will_ not, support "hooks" that aren't 
   actually used (and make sense) by normal kernel uses, for largely the 
   same reasons. Interfaces need to be architected, make sense, and have 
   real and valid GPL'd uses.

In other words, if a filesystem wants to do something fancy, it needs to 
do so WITH THE VFS LAYER, not as some plugin architecture of its own. We 
already have exactly the plugin interface we need, and it literally _is_ 
the VFS interfaces - you can plug in your own filesystems with 
"register_filesystem()", which in turn indirectly allows you to plug in 
your per-file and per-directory operations for things like lookup etc.

If that isn't enough, then the filesystem shouldn't make its own internal 
plug-in architecture that bypasses the VFS layer and exposes functionality 
that isn't necessarily sane. For example, reiser4 used to have (perhaps 
still does) these cool files that can be both directories and links, and I 
don't mind that at all, but I _do_ mind the fact that when Al Viro (long 
long ago) pointed out serious locking issues with them, those seemed to be 
totally brushed away.

So as far as I'm concerned, the problem with reiser4 is that it hasn't 
tried to work with the VFS people. Now, I realize that the main VFS people 
aren't always easy to work with (Al and Christoph, take a bow), but that 
doesn't really change the basic facts. Al in particular is _always_ right. 
I don't think I've ever had the cojones to argue with Al..

			Linus



From: Andrew Morton [email blocked]
Subject: Re: reiser4: maybe just fix bugs?
Date:	Tue, 1 Aug 2006 01:31:04 -0700

On Mon, 31 Jul 2006 10:26:55 +0100
"Denis Vlasenko" <vda.linux@googlemail.com> wrote:

> The reiser4 thread seem to be longer than usual.

Meanwhile here's poor old me trying to find another four hours to finish
reviewing the thing.

The writeout code is ugly, although that's largely due to a mismatch between
what reiser4 wants to do and what the VFS/MM expects it to do.  If it
works, we can live with it, although perhaps the VFS could be made smarter.

I'd say that resier4's major problem is the lack of xattrs, acls and
direct-io.  That's likely to significantly limit its vendor uptake.  (As
might the copyright assignment thing, but is that a kernel.org concern?)

The plugins appear to be wildly misnamed - they're just an internal
abstraction layer which permits later feature additions to be added in a
clean and safe manner.  Certainly not worth all this fuss.

Could I suggest that further technical critiques of reiser4 include a
file-and-line reference?  That should ease the load on vger.

Thanks.



From: Hans Reiser [email blocked]
Subject: Re: reiser4: maybe just fix bugs?
Date:	Mon, 31 Jul 2006 20:18:50 -0600

Andrew Morton wrote:

>On Mon, 31 Jul 2006 10:26:55 +0100
>"Denis Vlasenko" <vda.linux@googlemail.com> wrote:
>
>  
>
>>The reiser4 thread seem to be longer than usual.
>>    
>>
>
>Meanwhile here's poor old me trying to find another four hours to finish
>reviewing the thing.
>  
>
Thanks Andrew.

>The writeout code is ugly, although that's largely due to a mismatch between
>what reiser4 wants to do and what the VFS/MM expects it to do.
>
I agree --- both with it being ugly, and that being part of why.

>  If it
>works, we can live with it, although perhaps the VFS could be made smarter.
>  
>
I would be curious regarding any ideas on that.  Next time I read
through that code, I will keep in mind that you are open to making VFS
changes if it improves things, and I will try to get clever somehow and
send it by you.  Our squalloc code though is I must say the most
complicated and ugliest piece of code I ever worked on for which every
cumulative ugliness had a substantive performance advantage requiring us
to keep it.  If you spare yourself from reading that, it is
understandable to do so.

>I'd say that resier4's major problem is the lack of xattrs, acls and
>direct-io.  That's likely to significantly limit its vendor uptake.  (As
>might the copyright assignment thing, but is that a kernel.org concern?)
>  
>
Thanks to you and the batch write code, direct io support will now be
much easier to code, and it probably will get coded the soonest of those
features.  acls are on the todo list, but doing them right might require
solving a few additional issues (finishing the inheritance code, etc.)

>The plugins appear to be wildly misnamed - they're just an internal
>abstraction layer which permits later feature additions to be added in a
>clean and safe manner.  Certainly not worth all this fuss.
>
>Could I suggest that further technical critiques of reiser4 include a
>file-and-line reference?  That should ease the load on vger.
>
>Thanks.
>



From: Nick Piggin [email blocked]
Subject: Re: reiser4: maybe just fix bugs?
Date:	Tue, 01 Aug 2006 21:52:03 +1000

Andrew Morton wrote:
> On Mon, 31 Jul 2006 10:26:55 +0100
> "Denis Vlasenko" <vda.linux@googlemail.com> wrote:
> 
> 
>>The reiser4 thread seem to be longer than usual.
> 
> 
> Meanwhile here's poor old me trying to find another four hours to finish
> reviewing the thing.
> 
> The writeout code is ugly, although that's largely due to a mismatch between
> what reiser4 wants to do and what the VFS/MM expects it to do.  If it
> works, we can live with it, although perhaps the VFS could be made smarter.
> 
> I'd say that resier4's major problem is the lack of xattrs, acls and
> direct-io.  That's likely to significantly limit its vendor uptake.  (As
> might the copyright assignment thing, but is that a kernel.org concern?)
> 
> The plugins appear to be wildly misnamed - they're just an internal
> abstraction layer which permits later feature additions to be added in a
> clean and safe manner.  Certainly not worth all this fuss.
> 
> Could I suggest that further technical critiques of reiser4 include a
> file-and-line reference?  That should ease the load on vger.

I haven't really reviewed it, but when I grepped through it last, I
found a few alarming things, like use of __put_page, trying to remove
pages from pagecache (duplicating parts of vmscan.c, plus bugs), and
taking tree_lock.

Mostly didn't look like big problems to fix, but should be fixed for
mm/ maintainers' sanity. Maybe it's better now, though.

-- 
SUSE Labs, Novell Inc.
Send instant messages to your online friends http://au.messenger.yahoo.com 



From: Andrew Morton [email blocked]
Subject: Re: reiser4: maybe just fix bugs?
Date:	Tue, 1 Aug 2006 07:54:22 -0700

On Tue, 01 Aug 2006 21:52:03 +1000
Nick Piggin [email blocked] wrote:

> Andrew Morton wrote:
> > On Mon, 31 Jul 2006 10:26:55 +0100
> > "Denis Vlasenko" <vda.linux@googlemail.com> wrote:
> > 
> > 
> >>The reiser4 thread seem to be longer than usual.
> > 
> > 
> > Meanwhile here's poor old me trying to find another four hours to finish
> > reviewing the thing.
> > 
> > The writeout code is ugly, although that's largely due to a mismatch between
> > what reiser4 wants to do and what the VFS/MM expects it to do.  If it
> > works, we can live with it, although perhaps the VFS could be made smarter.
> > 
> > I'd say that resier4's major problem is the lack of xattrs, acls and
> > direct-io.  That's likely to significantly limit its vendor uptake.  (As
> > might the copyright assignment thing, but is that a kernel.org concern?)
> > 
> > The plugins appear to be wildly misnamed - they're just an internal
> > abstraction layer which permits later feature additions to be added in a
> > clean and safe manner.  Certainly not worth all this fuss.
> > 
> > Could I suggest that further technical critiques of reiser4 include a
> > file-and-line reference?  That should ease the load on vger.
> 
> I haven't really reviewed it, but when I grepped through it last, I
> found a few alarming things, like use of __put_page, trying to remove
> pages from pagecache (duplicating parts of vmscan.c, plus bugs), and
> taking tree_lock.

__put_page() has gone.

A lot of the VM duplication has gone.

tree_lock is still there a bit.  For two reasons:

a) A presently-unneeded duplication of the generic
   __set_page_dirty_nobuffers() and

b) Manipulation of the fake inode (I think).

   iirc the base problem here is that for whatever reason, the usual
   blockdev inode which filesystems use for metadata (ie: the pagecache
   which backs sb_bread(), etc) isn't suitable.  reiser4 wants to get a
   hold of its address_space ops, so it needs to create its own
   covers-all-the-partition, identify-mapped address_space.

> Mostly didn't look like big problems to fix, but should be fixed for
> mm/ maintainers' sanity. Maybe it's better now, though.

It's better.  More reviewing help is always appreciated ;)



From: Andi Kleen [email blocked]
Subject: Re: reiser4: maybe just fix bugs?
Date:	01 Aug 2006 21:32:16 +0200

Andrew Morton [email blocked] writes:

> On Mon, 31 Jul 2006 10:26:55 +0100
> "Denis Vlasenko" <vda.linux@googlemail.com> wrote:
> 
> > The reiser4 thread seem to be longer than usual.
> 
> Meanwhile here's poor old me trying to find another four hours to finish
> reviewing the thing.

I took a quick look at it and I must say that most of the things
that tripped me up when I first looked at it a long time ago
are gone now.

I guess there could be still quite a lot of cleanup, but it already
looks much better.

-Andi

• Archive of above thread
• KernelTrap interview with Andrew Morton
• KernelTrap interview with Nick Piggin
• KernelTrap interview with Hans Reiser