2008-09-01 02:37 -!- pgquiles(~pgquiles@153.Red-83-35-242.dynamicIP.rima-tde.net) has joined #tux3
2008-09-01 02:52 -!- pgquiles(~pgquiles@110.Red-83-41-45.dynamicIP.rima-tde.net) has joined #tux3
2008-09-01 03:01 -!- MaZe(~MaZe@c-24-6-86-168.hsd1.ca.comcast.net) has joined #tux3
2008-09-01 11:50 <shapor> flips: revision 199 broke inode.c, can pull fix from me 
2008-09-01 11:50 <flips> how broke?
2008-09-01 11:51 <flips> I only see 198 revisions in my repo
2008-09-01 11:57 <shapor> removed parameter from ext2_dump_entries
2008-09-01 11:57 <shapor> ah yeah revision numbers aren't global
2008-09-01 11:58 <flips> already fixed
2008-09-01 11:58 <flips> here
2008-09-01 11:58 <shapor> ok ;)
2008-09-01 11:58 <flips> the new interpreter makes it much easier to find bugs
2008-09-01 11:58 <flips> I found a bunch
2008-09-01 11:58 <flips> working on an inode table leaf split corruption one now
2008-09-01 11:59 <flips> the checking functions are really badly needed
2008-09-01 11:59 <flips> inode table block 0x0/40 (8c bytes free)
2008-09-01 11:59 <flips>   0x0: [0] mode 0000000 uid 0 gid 0 root 22:1 ctime 0 size 2000
2008-09-01 11:59 <flips>   0xd: [40] mode 0100700 uid 0 gid 0 root 24:1
2008-09-01 11:59 <flips>   0x27: [64] mode 0100700 uid 0 gid 0 root 27:1 ctime 0 size ffffffffffffff
2008-09-01 11:59 <flips> resize inum 0xd at 0x28 from 24 to 40
2008-09-01 11:59 <flips> inode table block 0x0/40 (7c bytes free)
2008-09-01 11:59 <flips>   0x0: [0] mode 0000000 uid 0 gid 0 root 22:1 ctime 0 size 2000
2008-09-01 11:59 <shapor> you haven't checked it in yet?
2008-09-01 11:59 <flips>   0xd: [40] mode 0100700 uid 0 gid 0 root 81c00000:24576
2008-09-01 11:59 <flips>   0x27: [80] mode 0100700 uid 0 gid 0 root 27:1 ctime 0 size ffffffffffffff
2008-09-01 11:59 <flips> not yet
2008-09-01 12:00 <flips> right after this bug
2008-09-01 12:00 <shapor> cool
2008-09-01 12:00 <flips> see the root attribute of inode d get messed up by the resize
2008-09-01 12:00 <shapor> ah
2008-09-01 12:00 <shapor> yeah
2008-09-01 12:01 <flips> for one thing, inode d isn't at offset 28, I don't know why it thinks it is
2008-09-01 12:01 <flips> anyway
2008-09-01 12:01 <flips> this one is my mess
2008-09-01 12:02 <flips> it turns out Tux3 can only do 64 petabytes with 256 byte blocks
2008-09-01 12:09 <shapor> thats it?!
2008-09-01 12:09 <flips> it's because the dump was printing in decimal :p
2008-09-01 12:12 <shapor> you mean pebibytes?
2008-09-01 12:12 <shapor> http://en.wikipedia.org/wiki/Petabyte
2008-09-01 12:13 <flips> 
ACTION doesn't subscribe to that hairy footed nonsense

2008-09-01 12:13 <shapor> why does anyone use base 10 anyway?
2008-09-01 12:14 <shapor> when describing these things
2008-09-01 12:14 <shapor> blame hard drive manufacturers i suppose
2008-09-01 12:16 <flips> why does anyone use base 10 for anything?
2008-09-01 12:16 <flips> something to do with counting on fingers and toes
2008-09-01 12:27 <flips> there was no bug
2008-09-01 12:28 <flips> intermediate state produced funny behavior
2008-09-01 12:40 <flips> on bug down
2008-09-01 12:40 <flips> parens around a conditional expression
2008-09-01 12:41 <flips> oh, that took care of two bugs
2008-09-01 12:41 <flips> nice
2008-09-01 12:41 <flips> ok time to check in
2008-09-01 12:41 <shapor> now thats efficient ;)
2008-09-01 12:43 <flips> ./tux3 read --seek 72057594037927930 foodev foo  <- this works
2008-09-01 12:43 <flips> reads the 64th petabyte of file foo in device foodev, with 256 byte blocks
2008-09-01 13:04 <pgquiles> according to this ( http://blogs.netapp.com/standards_watch/2007/12/emc-netapp-dona.html ), there should be an NDMP implementation available from SNIA, which would make it easier to implement NDMP support in strigi, but yesterday I was unable to find that source code :-?
2008-09-01 13:04 <pgquiles> oops, wrong channel
2008-09-01 13:04 <pgquiles> hi all, btw :-)
2008-09-01 13:04 <flips> :)
2008-09-01 13:05 <shapor> flips: going to add tux3 to the Makefile?
2008-09-01 13:05 <flips> not before I have a nap
2008-09-01 13:05 <flips> fee free
2008-09-01 13:05 <flips> feel free
2008-09-01 13:05 <flips> I'm writing a little post
2008-09-01 13:05 <shapor> paste the cmdline in your shell history to build it ;)
2008-09-01 13:05 <flips> which should help make a test
2008-09-01 13:06 <flips> yes
2008-09-01 13:06 <shapor> so i can add it without thinking as much 
2008-09-01 13:07 <flips> g99 -g -Wall -lpopt buffer.c diskio.c tux3.c -otux3
2008-09-01 14:02 <bh> hey
2008-09-01 14:02 <bh> flips: just came back last night from Burning Man
2008-09-01 14:03 <flips> burned out?
2008-09-01 14:03 <bh> eh ?
2008-09-01 14:03 <bh> no, I had a blast
2008-09-01 14:03 <flips> joke
2008-09-01 14:04 <bh> oh ok, yeah, I figured half of Google's infrastructure engineering went out there as well
2008-09-01 14:05 <flips> I checked in several thousands lines of patches while you were taking care of more important things ;-)
2008-09-01 14:06 <bh> nice, I have a lot of work todo but I'm half clueless about certain parts of the scheduler code
2008-09-01 14:07 <bh> gregory is coming up with fixes for various scheduler path issues, but I doubt that it's going to fix the latency problem. The cross locks are very problematic
2008-09-01 14:08 <flips> I'll get gone for a bit
2008-09-01 14:08 <bh> ok
2008-09-01 17:10 <flips> back
2008-09-01 18:36 -!- MaZe(~MaZe@c-24-6-86-168.hsd1.ca.comcast.net) has joined #tux3
2008-09-01 19:33 <konrad> flips: what does checking the integrity of leaf nodes involve?
2008-09-01 19:35 <konrad> also, gcc 4.3 won't build the tux3 tests
2008-09-01 20:20 <konrad> ld complains that /usr/lib/libpopt.so is incompatible
2008-09-01 20:20 <konrad> perhaps because tux3 is -std=gnu99?
2008-09-01 20:21 <konrad> oh wait
2008-09-01 20:21 <konrad> need to install 64-bit popt-devel
2008-09-01 20:23 <flips> konrad, hi
2008-09-01 20:24 <konrad> hi
2008-09-01 20:24 <flips> konrad, it would be a great start just to check that the entries are all in non-descending order
2008-09-01 20:24 <flips> for dleaf
2008-09-01 20:25 <flips> for both dealf and ileaf, the upside down dictionaries should contain offsets in non-descending order
2008-09-01 20:25 <flips> to bottom of the dictionary should not be below the top of the highest entry
2008-09-01 20:25 <flips> can get fancier from there, but that will already detect most corruption
2008-09-01 20:25 <konrad> ok
2008-09-01 20:26 <flips> :-)
2008-09-01 20:26 <flips> sounds like a hack about to begin
2008-09-01 20:26 <konrad> shh
2008-09-01 20:26 <flips> if that's what it takes :-)
2008-09-01 20:27 <konrad> before that
2008-09-01 20:27 <flips> another pretty straightforward project is to add more commands to tux3.c
2008-09-01 20:27 <konrad> I get some weird errors building tux3.c
2008-09-01 20:27 <flips> like "remove"
2008-09-01 20:27 <flips> ah
2008-09-01 20:27 <konrad> for some reason references to the inline functions go through ld
2008-09-01 20:27 <flips> you need to build with gcc -std=gnu99
2008-09-01 20:27 <konrad> I am
2008-09-01 20:27 <flips> the errors are?
2008-09-01 20:28 <flips> 
ACTION checks to see if make works

2008-09-01 20:28 <konrad> tux3/user/test/iattr.c:95: undefined reference to `encode16'
2008-09-01 20:28 <konrad> x10
2008-09-01 20:28 <flips> you
2008-09-01 20:28 <flips> um
2008-09-01 20:28 <konrad> lines 98, 99, 100, 103, 104, 107, 111, 114
2008-09-01 20:29 <konrad> and 128, ...
2008-09-01 20:29 <konrad> some others
2008-09-01 20:29 <flips> that is odd
2008-09-01 20:29 <flips> check your compile output
2008-09-01 20:29 <konrad> there are some warnings about iattr.c: In function ‘decode16’:
2008-09-01 20:29 <konrad> iattr.c:20: warning: ‘be_to_u16’ is static but used in inline function ‘decode16’ which is not static
2008-09-01 20:29 <flips> just a sec
2008-09-01 20:29 <flips> ah
2008-09-01 20:29 <flips> interesting
2008-09-01 20:29 <flips> ok
2008-09-01 20:29 <flips> what it to static inline
2008-09-01 20:30 <flips> change it to static inline
2008-09-01 20:30 <flips> I'll do that right now
2008-09-01 20:30 <konrad> k
2008-09-01 20:30 <konrad> should I do it to, or will you tell me when to pull?
2008-09-01 20:30 <flips> just about done
2008-09-01 20:31 <konrad> builds now
2008-09-01 20:31 <flips> updated in repo
2008-09-01 20:31 <flips> good
2008-09-01 20:31 <flips> what's your gcc version?
2008-09-01 20:31 <flips> gcc --version
2008-09-01 20:31 <konrad> 4.3.0
2008-09-01 20:31 <konrad> 20080428
2008-09-01 20:31 <flips> I'm 4.1.2
2008-09-01 20:32 <flips> looks like a gcc regression
2008-09-01 20:32 <flips> smells like
2008-09-01 20:32 <konrad> certainly possible
2008-09-01 20:32 <flips> but I'
2008-09-01 20:32 <flips> but I'm ok with this resolution
2008-09-01 20:32 <flips> I'm pretty happy with those endian macros
2008-09-01 20:32 <flips> very efficiently implemented
2008-09-01 20:32 <konrad> mhm
2008-09-01 20:32 <flips> didn't even know they were there until last week
2008-09-01 20:33 <flips> found them by accident
2008-09-01 20:33 <flips> really essential
2008-09-01 20:33 <flips> #include <byteswap.h> <- the magic words
2008-09-01 20:34 <konrad> yep
2008-09-01 20:34 <konrad> I'm looking at that right now
2008-09-01 20:34 <konrad>  29 /* Return a value with all bytes in the 16 bit argument swapped.  */
2008-09-01 20:34 <konrad>  30 #define bswap_16(x) __bswap_16 (x)
2008-09-01 20:34 <konrad> does it do-the-right-thing on big endian archs?
2008-09-01 20:34 <flips> you just did your first bug hunt ;-)
2008-09-01 20:34 <konrad> yay
2008-09-01 20:35 <konrad> I still get a couple format string warnings
2008-09-01 20:35 <flips> bswap is a 2 byte asm instruction as I recall
2008-09-01 20:35 <flips> runs at superscaler speed these days I think - can do more than one bswap per cycle
2008-09-01 20:35 <konrad> what about on ppc, m68k or other BE archs?
2008-09-01 20:35 <flips> in other words, as close to free as it gets
2008-09-01 20:35 <konrad> does it omit those or still try to swap?
2008-09-01 20:36 <flips> there are similar instructions on some of the other arches
2008-09-01 20:36 <flips> but since the native resolution for tux3 is bigendian, ppc is fine
2008-09-01 20:36 <flips> no swapping to do
2008-09-01 20:36 <konrad> k
2008-09-01 20:36 <flips> big endian is _way_ nicer for debugging
2008-09-01 20:37 <konrad> big endian is way nicer for everything :)
2008-09-01 20:37 <konrad> bummer x86 isn't big endian
2008-09-01 20:37 <flips> had to stare at the hexdumps sometimes for a while, wondering why the lsb was up at the high end of the struct, I'm so used to the braindamaged intel order
2008-09-01 20:37 <flips> x86 is most braindamage, that happened to be implemented better than the other guys
2008-09-01 20:37 <flips> too bad about that
2008-09-01 20:38 <flips> motorola really blew it
2008-09-01 20:38 <flips> ibm's doing ok with ppc
2008-09-01 20:38 <flips> but I'd like to see them do more on power efficiency
2008-09-01 20:38 <flips> ppc rules the console world
2008-09-01 20:38 <flips> which is getting to be more machines than the biz world even
2008-09-01 20:39 <flips> more than certain search engine operators even
2008-09-01 20:41 <konrad> Are:
2008-09-01 20:41 <konrad> inode.c:421: warning: format ‘%Lx’ expects type ‘long long unsigned int’, but argument 2 has type ‘block_t’
2008-09-01 20:41 <konrad> tux3.c:139: warning: format ‘%Li’ expects type ‘long long int’, but argument 2 has type ‘u64’
2008-09-01 20:41 <konrad> tux3.c:175: warning: format ‘%Li’ expects type ‘long long int’, but argument 2 has type ‘u64’
2008-09-01 20:41 <konrad> supposed to happen?
2008-09-01 20:46 <flips> cast the printf arg using (L)
2008-09-01 20:47 <flips> that's a tux3 macro, same as (long long unsigned)
2008-09-01 20:47 <flips> I'm running 32 bit here, so you need to post your hg patch to the mailing list
2008-09-01 20:48 <flips> 
ACTION has to get his hammer machine online

2008-09-01 20:50 <konrad> right
2008-09-01 20:51 <konrad> that's what I thought, so I made the patch already
2008-09-01 20:51 <konrad> was just waiting on posting it 
2008-09-01 20:55 <konrad> tux3 will need to go GPLv2 at some point to get into the kernel, no?
2008-09-01 21:02 <flips> applied, thanks
2008-09-01 21:03 <flips> that is correct, there is a post about that
2008-09-01 21:04 <flips> I reserve the right to relicense tux3, including the downgrade to v2 for the kernel port
2008-09-01 21:04 <flips> I suppose I should ping Eben and see if he likes my slight hack of his license ;-)
2008-09-01 21:16 -!- caoliver(~oliver@75-134-208-20.dhcp.trcy.mi.charter.com) has joined #tux3
2008-09-01 21:16 -!- caoliver(~oliver@75-134-208-20.dhcp.trcy.mi.charter.com) has left #tux3
2008-09-01 21:51 <konrad> flips: what are the offsets?
2008-09-01 21:51 <flips> offsets?
2008-09-01 21:51 <flips> oh
2008-09-01 21:51 <konrad> in dleaf
2008-09-01 21:51 <flips> 16 bit address within a dleaf
2008-09-01 21:51 <konrad> looks like there's one per entry
2008-09-01 21:51 <konrad> ok
2008-09-01 21:51 <flips> or 16 bit offset from the beginning of data in an ileaf
2008-09-01 21:52 <flips> ileaf would be an easier place to start
2008-09-01 21:52 <flips> dleaf has a two level index, both levels upside down, which is a little confusing
2008-09-01 21:52 <konrad> heh
2008-09-01 21:53 <flips> the other confusing detail is that the 0th index entry is not actually represented, it is assumed to be zero
2008-09-01 21:53 <flips> for both dleaf and ileaf
2008-09-01 21:54 <flips> I think I might be able to code an inline to make that a little clearer
2008-09-01 22:01 <flips> another complication for ileaf is that leaf->count is allowed to be zero
2008-09-01 22:02 <flips> that means that dick[-i] can be invalid either because i is zero or leaf->count is zero, which usually implies the same, but not for ileaf
2008-09-01 22:02 <flips> which allows offsets higher than leaf->count
2008-09-01 22:03 <flips> because the ileaf dictionary can be extended to accomodate.
2008-09-01 22:57 <konrad> how is the size of the zeroth inode an ileaf found?
2008-09-01 22:59 <konrad> inode of an ileaf*
2008-09-01 23:01 <konrad> er, if leaf->count is zero
2008-09-01 23:02 -!- MaZe(~MaZe@c-24-6-86-168.hsd1.ca.comcast.net) has joined #tux3
2008-09-01 23:07 <konrad> http://pastie.org/264354.txt <-- does that look about right?
2008-09-01 23:17 <konrad> er
2008-09-01 23:18 <konrad> I guess dict[-1] exists even if there's only the zeroth inum
2008-09-01 23:18 <konrad> therefor dict[-btree->entries_per_leaf - 1] is part of the dict too