Build extra utilities for the HP Media Vault V1 (mv20xx)

hosted by ibiblio

Document changes:

  1. 23 Nov 2008 - Original posting.
  2. 15 Feb 2009 - Status update on CUPS, no further progress, minor wording fixes.
  3. 12 Nov 2009 - Added lsMVdiskinfo (see last of Notes just below.)
  4. 15 Nov 2009 - Added rsync 3.0.6 (see utilities table below).
  5. 23 May 2010 - Added note on PATH order; updated UID synchronization; made other minor changes.
  6. 11 Sep 2010 - Added PCIutils for 'lspci'; logout/in note for MacOS UID change; other minor additions.

Important Accolades, Notes and Caveats:

  • The foundation of this work is the excellent and essential work already done by Lee Devlin (and the others he's linked to) described on his HP Media Vault FAQ/Knowledgebase (HPMV-FAQ). You will need to download the cross-compiler tools linked on that FAQ. Many of the same caveats it has (especially "where you see a hostname substitute your own") apply to this HOWTO as well.
  • My mv2010 is setup to run in "user mode" not "workgroup mode" although this isn't strictly necessary. Some of the things you'll see below though, are based upon this setup, and you have to get a non-MD5 passwd hash for root from somewhere (I copied the 13 character hash from my user's password entry to root's).
  • The telnet capability mentioned in the HPMV-FAQ is essential to accomplishing this, so you must be comfortable with a really basic Unix command line (as you likely are if you're reading this).
  • The utilities operate out of /mnt, which, on my mv2010 is a "bind" mount of a share named "MntLocal". This "bind" mount is executed by the /shares/Volume1/ script described in the HPMV-FAQ. I chose /mnt as it does not seem to be used elsewhere in the mv20xx, as opposed to /opt which holds the MediaServer or /usr/local which has Samba.
  • My original build machine was running Centos-5.2 x86 with a base installation of "workstation". I currently use Ubuntu Desktop 9.10 x86. I run the command: 'mount hpmediavault:/shares/Volume1/MntLocal /mnt', so that a 'make install' goes to the correct spot on the build machine and on the mv20xx. I have not tested 64-bit Linux build environments.
  • I've added a small 'C' program, named lsMVdiskinfo which lists the disk header and partition information of an mv20xx disk. You can find a compressed tar file of it, its GPL, and a README, HERE. This builds on Linux, MacOS and Cygwin/Windows; and was tested on Ubuntu 9.XX/10.XX, RedHat ES 5.4, MacOS 10.6, Cygwin 1.7.X and NASoC Linux (the run-time environment whose setup follows).

Rationale (or what I wanted to accomplish):

I originally wanted two things, sshd with full login instead of telnetd, and IPP for bidirectional printer handling. I also wanted a few more of the usual utilities for scripting my startup script, and in trying to investigate the mv2010, I found that 'find', 'lsof' and a recursive capable 'ls' are all useful. Generally, I wanted to login and be able to look around without too many hurdles. I also wanted to move many files, volume to volume (to prepare for RAID1). I tried the 'jrun' stuff linked to within the FAQ, but did not like the 'chroot jail' in which it operated (although my setup is a bit more dangerous, then, if run as root, but most of the main partition is read-only anyhow).

I also wanted to accomplish this for myself, which is why I did not look at the email groups or anything much beyond the HPMV-FAQ and the content it linked to. This is why I may have duplicated work that was already done, but I don't mind. I also did not set things up exactly as suggested in the adding software section of the HPMV-FAQ, preferring shorter path names.

What's been built so far:

Since the build tool chain for the mv20xx is relatively old, I have had to pick older releases of some of these utilities. In the build attempts with newer releases of those, there were several failures, especially with large '#define' values. I also had to avoid or configure around some utilities which included C++, as that's not completely setup in the tool chain. Along with the utility names and release levels below, I've linked the site where I found the source code. Most everything needs a minor nudge or two to build in the cross-compiled environment, so I've included either the patch I made (usually to Makefile), and/or the incantation for the utility's configure script.

Limited RAM is an issue on the mv20xx. In retrospect, and in practice, while cpio is useful for clean inter-disk transfers, building tar and the two compression utilities might be overkill, as they, especially bzip, can be very slow on this machine with its meager 64K RAM (and possibly somewhat scrawny CPU :-)). The compression utilities do work though (tested with -t from another machine, and vice versa).

You'll notice that CUPS (for IPP) is not included below, yet. Versions 1.2 and beyond require DBus which is not in the firmware. Version 1.1 contains C++ code, which is not fully supported by the build environment. So, I'm working with version 1.0.5 which has issues building in a cross-compiled environment. Status on this remains unchanged as of: 11 Sep 2010.

Patch files are installed with 'patch -p0 < patchfile' (usually -p0; it depends on your cwd; might be -p1). The configure scripts are executed using '.' (from bash; YMMV when using other shells). Many of the GNU utilities share the same configure script. Once the patch and/or configure is done 'make install' usually does what you want. You might want to become root for a separate 'make install' in order to setuid an executable, but so far, only 'su' in the coreutils distribution and 'ssh' (for split privilege directory ownership) would require it, and if you're logging into the mv20xx as the root user, setuid isn't necessary (nor is 'su' at that point, for the most part).

Utility-Release Patch Configure Comment
ex-050325 patch   The original, small BSD editor, Vi (ex). Copy your /etc/termcap to /mnt/etc.
bzip2-1.0.5 patch   A bit slow in execution.
coreutils-5.97   configure Uses
cpio-2.9   configure Uses
findutils-4.1.20   configure Uses
gawk-3.1.5   configure Uses
grep-2.5.1   configure Disabled 'perl-regular-expression' library in configure.
gzip-1.3.9   configure Uses
lsof_4.77 patch configure  
net-tools-1.60 patch configure Configured only AFUNIX, AFINET, Ethernet; all others unset including IPv6.
openssh-5.1p1   configure Depends on openssl. See Configure section #2 below for more setup.
openssl-0.9.8i patch configure Some errors in 'engine' link. No crypto PCI cards in my mv2010, so ignored.
pciutils-3.0.0 patch   Patches Makefile; Have /mnt mounted for (already built) zlib access.
procps-3.2.6 patch   Skips top, slabtop and watch, as they need ncurses, as yet unbuilt.
rsync-3.0.6   configure Uses
tar-1.15   configure Uses
zlib-1.2.3   configure Both openssl and openssh depend upon zlib.
mv20xx boot My script and the script it checks for and calls if found and executable.

Steps needed to do the build:

  1. Create a share on your mv20xx enabling NFS mounting for it. I named mine MntLocal.
  2. Pick a directory, e.g. ~PUT_YOUR_USERNAME_HERE/src, where you will do your work. Create it if necessary. Below, I've shortened PUT_YOUR_USERNAME_HERE to PYUH. I'll refer to ~PYUH/src below and you should adjust any of those instructions to match the directory you chose.
  3. Download the firmware build tools and source tree from the HPMV-FAQ site.
  4. Change to that directory and untar the two archives; Oxford_GPL_1359.tar.gz first, then Oxford_GPL_1359-K03_1015.tar.gz next.
  5. To mirror what I've setup, briefly become root on your build system to:
    1. cd /opt
    2. ln -s ~PYUH/src/Oxford_GPL_1359/brcm
    3. ln -s ~PYUH/src/Oxford_GPL_1359/nasoc
    4. mount hpmediavault:/shares/Volume1/MntLocal /mnt
    5. chown PYUH /mnt
    NFS mounting (as opposed to SMB mounting) is useful, since the mv20xx allows the build system's root user to actually be the root user on the filesystem (as opposed to the 'nobody' user). Exit the root shell for now. The symlinks match my scripts (the first symlink also matches the setup in the firmware README, the second one makes the firmware instructions a bit easier by substituting /opt for `pwd`).
  6. Again, mirroring the firmware README, make the additions to your PATH variable. For the bash shell the following will work: 'export PATH=/opt/nasoc/tools:/opt/brcm/hndtools-mipsel-linux/bin:$PATH'. Note that this is the opposite order from the firmware setup, whose make files are already prepared for the cross-compile environment. This PATH order allows just 'gcc' to find, effectively, 'mipsel-linux-gcc'. [This can cause its own issues though, as you might have to do a build in two stages if the build process includes intermediate tools, i.e. ones which have to run on the build host to create a file necesary for the final target build.]
  7. Setup the /mnt/lib directory by doing:
    1. cd /opt/brcm/hndtools-mipsel-linux-3.0/mipsel-linux
    2. find lib -print | cpio -pdvm /mnt'
  8. Setup the other directories: 'mkdir -p /mnt/{etc,bin,sbin,share,info,man/man{1,5,8},usr/bin}'.
  9. Back in your build directory, pick a utility, download it, untar it and 'cd' into the newly untarred directory. Try a GNU one (I'd suggest 'find') first.
  10. Run the configure script for it (and/or patch first if necessary), and then 'make install'.
  11. Telnet to your mv20xx, and do 'mount -o bind /shares/Volume1/MntLocal /mnt'.
  12. Still on the mv20xx, set the LD_LIBRARY_PATH variable to reference /mnt/lib first and adjust your PATH:
    1. export LD_LIBRARY_PATH=/mnt/lib:/usr/lib:/lib
    2. export PATH=/mnt/bin:$PATH
    Then test out your newly built utility!

How to configure some things on the mv20xx.

  1. Synchronizing UIDs across systems.

    When using NFS, I wanted to make sure that all the files were owned by the same UID, so that access from a Mac and from a Linux system would be relatively seamless. I use Windows Services for Unix (SFU) with the password file generated by Cygwin as the mapping file in order to have all the systems mount the mv2010 shares via NFS with the same UID.

    I decided that Windows was the most difficult system on which to change the UID, and so it would be the one driving the value on the other systems. While the UID is a long string on Windows, the Cygwin password file gives a mapping which provides a valid Unix UID, and the SFU mapper process is setup to use that file. That UID became the number to establish on the other systems.

    Here's some of the things I did to have them match and stay matching:

    On the mv20xx (see

    • Set the nvram values related to uid and gid.
    • Modify the passwd and group files to better match those of my MacOS and Linux systems, and copy my versions into place every mv20xx startup, just in case.
    • Took the password generated for my user (from user mode setup in the mv20xx browser gui), and copied it to the root user to give the root user a known password. This is the second field of the password file, on the mv20xxx it's the 13 characters after the first ':' (when a password exists, otherwise you might see an 'x' or an asterisk). Note that ':' is the field separator in the password file, so the password hash is the second field in each row. The username is the first field in each row.

    On a Linux system:

    This is the easiest to change. Most Linux distributions provide a menu selection from the login screen that puts the X-windows system aside briefly, allowing a terminal login, in this case login as the root user. Once root, you can do the following two commands and logout:

    • Run 'vipw' to edit the password file, and change the third field of the line containing your username to the decided upon UID, say 501 or 1040. Save your changes (':wq!' will do). You must reserve the number that was originally there for the next command.
    • Now find all the files owned by the old UID and set them to the new one. A command that will work on many systems is "find / -xdev -user OLDUID -exec chown -h NEWUID '{}' \;". I throw '-xdev' in there in case there are any network or CD/DVD filesystems mounted on your system already. The 'chown' command changes the owner, and '-h' says "change the owner of any symbolic links themselves, not the file linked to" while the the curly-braces represent the file found, and the single-quotes around them avoids errors on space characters in filenames. If '/home' is a separate partition on your system, you'll have to run the command again, substituting '/home' for '/'.

    On a MacOS system:

    This is only a bit more difficult to change as the UID change is not a simple edit session. Here's the two commands to perform as root, in a Terminal window, via 'su' or 'sudo':

    • Change the UID using the "directory service command line" utility. In a terminal window as root enter in 'dscl .'. At the prompt enter 'cd Users', then 'cat PYUH' to find out the current UniqueID, then '-change PYUH UniqueID OLDUID NEWUID' to change it, then 'quit'. Again, reserve the OLDUID for the next command(s).
    • The same second command ('find ...') as for the Linux system above; repeating, replacing '/' with '/Users', if that's a separate partition on your system.

    You should logout and then log back in to make sure the change is picked up by the Finder and other programs.

  2. Setting up openssh on the mv20xx.

    The Makefile for openssh attempts to create the system's keys and such, but (of course) cannot run the MIPS executable it just made in order to do so. You'll have to do some of this one-time setup by hand. On my system that would be:

    • /mnt/bin/ssh-keygen -t rsa1 -f /mnt/etc/ssh_host_key -N ""
    • /mnt/bin/ssh-keygen -t dsa -f /mnt/etc/ssh_host_dsa_key -N ""
    • /mnt/bin/ssh-keygen -t rsa -f /mnt/etc/ssh_host_rsa_key -N ""
    • /mnt/sbin/sshd -t -f /mnt/etc/sshd_config # this tests the configuration.

    I'll leave it to others to figure out the setup needed for 'sftp' for example.

This is still a work in progress. Please feel free to provide any constructive feedback that would improve it.