Kaboing

Kaboing is a ping PogoPlug I picked up in September, 2011; they were on sale for $40, and it was too good to pass up. I'm adding notes here on its setup; my eventual plan/hope is to host this website and my wife's on it.

I followed these instructions, and aside from mistaking which IP address the plug got (and thus trying to SSH as root into my laptop) everything went fine.

USB problems

...and then I ran into USB problems. For some reason, the USB stick I'd been using as a root FS was no longer booting, and I'd get the Pogoplug environment. This was a stick I'd had lying around, so I tried another stick (Lexar 8GB) and had the same problems.

This page has a list of known-good/-bad sticks, but the Lexar wasn't listed there; it did have a link to this troubleshooting thread, though. Seems that some USB sticks take longer to initialize than others, and the board decides they're not working. I tried setting the boot delay to 20 seconds instead of 10:

/usr/sbin/fw_setenv usb_rootdelay 20

and rebooted (warm). That didn't work. I'm starting to suspect that it's the external hard drive that's causing problems, possibly because it's slow starting up; when I remove it, the machine seems to reboot into Debian just fine. (This forum posting is very similar.)

After some testing, I think what's going on is that the installation script from Jeff tries to boot from /dev/sda1 -- and the devices attached (there are a few of them!) are getting enumerated differently each time. One way around this would be to attach one and only one disk, but that's not very fun; another would be to tell the Pogoplug to boot from a UUID or an e2labelled-filesystem rather than a device entry.

I tried telling it to boot from a label:

/usr/sbin/fw_setenv usb_init "run usb_scan ; setenv usb_root LABEL=ROOTFS"

That didn't work. Tried setting:

/usr/sbinfw_setenv usb_init 'usb stop; usb start; run usb_scan; setenv usb_root LABEL=ROOTFS'

Still no; same pattern (see description below)

Netconsole

These instructions helped immensely:

fw_setenv serverip [my laptop ip]
fw_setenv ipaddr [kaboing's ip]
fw_setenv if_netconsole 'ping $serverip'
fw_setenv start_netconsole 'setenv ncip $serverip; setenv bootdelay 10; setenv stdin nc; setenv stdout nc; setenv stderr nc; version;'
fw_setenv preboot 'run if_netconsole start_netconsole'

In Debian, fw_setenv is at /usr/sbin. I can listen with socat like so:

socat STDIO UDP-LISTEN:6666

Problem: Pogoplug goes back to default environment every other boot

When I have a USB hard drive plugged into the Pink Pogoplug (front USB port; have not yet tried other ports), it will boot into Debian every other boot; the rest of the time, it'll boot back into the default Pogoplug Linux environment. I even tried taking out all the USB flash drives except one (I had three), and leaving in the hard drive, and it still craps out in this way.

The version of UBoot I have installed is 2010.09.

The error message

Here's the error message I see:

(Re)start USB...
USB:   Register 10011 NbrPorts 1
USB EHCI 1.00
scanning bus for devices... 4 USB Device(s) found
scanning bus for storage devices... error in inquiry

whereas on a successful boot it looks like this:

USB:   Register 10011 NbrPorts 1
USB EHCI 1.00
scanning bus for devices... 4 USB Device(s) found
   scanning bus for storage devices... 2 Storage Device(s) found
Loading file "/boot/uImage" from usb device 0:1 (usbda1)

Other reports

This has been noted in a few places:

This forum post on Jeff Doozan's site
Here, too, including mention of a possible fix (which didn't work for me; see below)
Another forum post on PlugComputer.org's forums
This post mentions a similar problem with a Porsche Lacie hard drive, which is what I have. Theirs is USB-powered, though, and mine has an external power supply. However, I also have this problem if I disconnect the external HD entirely.
This Debian bug report

Pattern is that it'll find Linux every second boot, which makes this easy to test...first it works; reboot and it doesn't; reboot and it does...rinse and repeat.

This workaround is to stick the kernel and initrd on an MMC card, which has its own initialization routines -- quite separate from the USB stuff.

Attempts to fix

What has not worked for me:

telling it to boot by label (as noted above)
Adding a boot delay to let things settle down (as noted above)
Getting uboot to initialize USB twice (as noted above)
Resetting mtd0 (it told me I already had the latest mtd0, and offered to reset the settings; that didn't work either)
This change to the bootcmd...in fact, it somehow borked the boot process entirely.

When working, the netconsole shows this:

(Re)start USB...
USB:   Register 10011 NbrPorts 1
USB EHCI 1.00
scanning bus for devices... 4 USB Device(s) found
   scanning bus for storage devices... 1 Storage Device(s) found
Loading file "/boot/uImage" from usb device 0:1 (usbda1)
1 bytes read
Found bootable drive on usb 0:1
Loading file "/boot/uImage" from usb device 0:1 (usbda1)
1435184 bytes read
Loading file "/boot/uInitrd" from usb device 0:1 (usbda1)
4528474 bytes read

When not working, it looks like this:

(Re)start USB...
USB:   Register 10011 NbrPorts 1
USB EHCI 1.00
scanning bus for devices... 4 USB Device(s) found
   scanning bus for storage devices... error in inquiry
0 Storage Device(s) found
** Block device usb 0 not supported
** Block device usb 1 not supported
** Block device usb 2 not supported
** Block device usb 3 not supported
** Block device usb 0 not supported
** Block device usb 0 not supported
Wrong Image Format for bootm command
ERROR: can't get kernel image!
stopping USB..

Note how it shows that 0 storage devices are found. (When I had three flash drives & the hard drive, every other boot would find an off-by-one number of devices.)

Occasionally I also see:

scanning bus for devices... 4 USB Device(s) found
   scanning bus for storage devices... error in inquiry

when it's trying to enumerate the storage during boot time.

nginx and php5

Some benchmarking w/my wife's blog:

Current server (Apache2, 2.8GHz P4, 1.5 GB RAM):

$ ab -n 100 -c 10 http://torturedpotato.com/cheeseblog/ This is ApacheBench, Version 2.3 <$Revision: 655654 $> Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ Licensed to The Apache Software Foundation, http://www.apache.org/



Benchmarking torturedpotato.com (be patient).....done


Server Software:        Apache/2.2.16
Server Hostname:        torturedpotato.com
Server Port:            80

Document Path:          /cheeseblog/
Document Length:        45143 bytes

Concurrency Level:      10
Time taken for tests:   41.481 seconds
Complete requests:      100
Failed requests:        0
Write errors:           0
Total transferred:      4541000 bytes
HTML transferred:       4514300 bytes
Requests per second:    2.41 [#/sec] (mean)
Time per request:       4148.117 [ms] (mean)
Time per request:       414.812 [ms] (mean, across all concurrent
requests)
Transfer rate:          106.91 [Kbytes/sec] received

Connection Times (ms)
      min  mean[+/-sd] median   max
Connect:        0    3   9.7      0      49
Processing:  3719 4128 136.3   4114    4499
Waiting:     3518 3918 123.4   3895    4270
Total:       3719 4131 139.3   4117    4548

Percentage of the requests served within a certain time (ms)
  50%   4117
  66%   4156
  75%   4175
  80%   4201
  90%   4348
  95%   4434
  98%   4485
  99%   4548
 100%   4548 (longest request)

1 PHP5 process:

$ ab -n 100 -c 10 http://kaboing.saintaardvarkthecarpeted.com/ This is ApacheBench, Version 2.3 <$Revision: 655654 $> Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ Licensed to The Apache Software Foundation, http://www.apache.org/



Benchmarking kaboing.saintaardvarkthecarpeted.com (be
patient).....done


Server Software:        nginx/0.7.67
Server Hostname:        kaboing.saintaardvarkthecarpeted.com
Server Port:            80

Document Path:          /
Document Length:        46256 bytes

Concurrency Level:      10
Time taken for tests:   219.814 seconds
Complete requests:      100
Failed requests:        0
Write errors:           0
Total transferred:      4649700 bytes
HTML transferred:       4625600 bytes
Requests per second:    0.45 [#/sec] (mean)
Time per request:       21981.435 [ms] (mean)
Time per request:       2198.144 [ms] (mean, across all concurrent
requests)
Transfer rate:          20.66 [Kbytes/sec] received

Connection Times (ms)
      min  mean[+/-sd] median   max
Connect:        0    0   0.1      0       1
Processing:  2185 21004 3682.1  21759   24027
Waiting:     2059 20876 3682.0  21630   23901
Total:       2185 21004 3682.0  21759   24027

Percentage of the requests served within a certain time (ms)
  50%  21759
  66%  21764
  75%  21774
  80%  21775
  90%  24004
  95%  24011
  98%  24024
  99%  24027
 100%  24027 (longest request)

4 PHP5 processes (fcgi-spawn -C 4):



Benchmarking kaboing.saintaardvarkthecarpeted.com (be
patient).....done


Server Software:        nginx/0.7.67
Server Hostname:        kaboing.saintaardvarkthecarpeted.com
Server Port:            80

Document Path:          /
Document Length:        46256 bytes

Concurrency Level:      10
Time taken for tests:   229.306 seconds
Complete requests:      100
Failed requests:        0
Write errors:           0
Total transferred:      4649700 bytes
HTML transferred:       4625600 bytes
Requests per second:    0.44 [#/sec] (mean)
Time per request:       22930.621 [ms] (mean)
Time per request:       2293.062 [ms] (mean, across all concurrent
requests)
Transfer rate:          19.80 [Kbytes/sec] received

Connection Times (ms)
      min  mean[+/-sd] median   max
Connect:        0    0   0.1      0       1
Processing:  8829 22208 3640.0  22339   30301
Waiting:     8319 21703 3636.9  21820   29792
Total:       8829 22208 3640.0  22339   30301

Percentage of the requests served within a certain time (ms)
  50%  22339
  66%  23381
  75%  23790
  80%  24030
  90%  26219
  95%  26737
  98%  30280
  99%  30301
 100%  30301 (longest request)

4 plus WP Super Cache



Benchmarking kaboing.saintaardvarkthecarpeted.com (be
patient).....done


Server Software:        nginx/0.7.67
Server Hostname:        kaboing.saintaardvarkthecarpeted.com
Server Port:            80

Document Path:          /
Document Length:        46764 bytes

Concurrency Level:      10
Time taken for tests:   10.097 seconds
Complete requests:      100
Failed requests:        0
Write errors:           0
Total transferred:      4706000 bytes
HTML transferred:       4676400 bytes
Requests per second:    9.90 [#/sec] (mean)
Time per request:       1009.714 [ms] (mean)
Time per request:       100.971 [ms] (mean, across all concurrent
requests)
Transfer rate:          455.15 [Kbytes/sec] received

Connection Times (ms)
      min  mean[+/-sd] median   max
Connect:        0    0   0.1      0       1
Processing:   371  978 155.1   1000    1198
Waiting:      358  945 154.1    969    1176
Total:        372  978 155.0   1000    1199

Percentage of the requests served within a certain time (ms)
  50%   1000
  66%   1050
  75%   1080
  80%   1100
  90%   1130
  95%   1167
  98%   1187
  99%   1199
 100%   1199 (longest request)

External Links

Notes on a SheevaPlug -- strong environmental/energy use focus. Oh, and he runs a gallery web app in Java on his plug.
Plug Computer FAQ
Setting up netconsole on a pogoplug -- I need to do this