/opt/csw/lib/cups/filter/pstopxl /opt/csw/lib/cups/filter/pstoraster
Over the last two days, in a frenzy of activity, I got some awesome done at work: using git and Vagrant, I finally got Cfengine to install packages in Ubuntu without asking me any goram questions. There were two bits involved:
Telling Apt to use old config files. This prevents it from asking for confirmation when it comes across your already-installed-with-Cfengine config file. Cfengine doesn't do things in a particular order, and in any case I do package installation once an hour -- so I might well have an NTP config file show up before the NTP package itself.
Preseeding, which I've known about for a while but have not had a chance to get right. My summer student came up with a script to do this, and I hope to be able to release it.
Now: Fully automated package installation FTMFW.
And did you know that Emacs can check your laptop battery status? I didn't.
Excellent overview of one workflow on Stack Overflow.
Mandriva shows how to set up rpm macros to use git during rpmbuild.
Software project A is kept in git. A depends on unrelated project B, so B is configured as a submodule in A. Unfortunately, A's Makefile assumes B's location in that subdirectory, with no provision for looking for its libraries and header files in the usual locations. Good or whack? @FunnelFiasco: "I concur. It's total bullshit."
Unrelated: xpdf crashes on Ubuntu 12.04. Wow.
A nice thing about working at a university is that you get all this time off at Xmas, which is really nice; however, it's also the best possible time to do all the stuff you've been saving up. Last year my time was split between this job and my last; now, the time's all mine, baby.
Today will be my last of three days in a row where the machines have been all mine to play with^W^Wupgrade. I've been able to twiddle the firewall's NIC settings, upgrade CentOS using Cfengine, and set up a new LDAP server using Cobbler and CentOS Directory Server. I've tested our UPS' ATS, but discovered that NUT is different from APCUPSD in one important way: it doesn't easily allow you to say "shut down now, even though there's 95% battery left". I may have to leave testing of that for another day.
It hasn't all gone smoothly, but I've accomplished almost all the important things. This is a nice surprise; I'm always hesistant when I estimate how long something will take, because I feel like I have no way of knowing in advance (interruptions, unexpected obstacles...you know the drill). In this case, the time estimates for individual tasks were, in fact, 'way paranoid, but that gave me the buffer that I needed.
One example: after upgrading CentOS, two of our three servers attached to StorageTek 2500 disk arrays reported problems with the disks. Upon closer inspection, they were reporting problems with half of the LUNs that the array was presenting to them -- and they were reporting them in different ways. It had been a year or longer since I'd set them up, and my documentation was pretty damn slim, so it took me a while to figure it out. (Had to sleep on it, even.)
The servers have dual paths to the arrays. In Linux, the multipath drivers don't work so well with these, so we used the Sun drivers instead. But:
cfservd
had refused its connection because I had the MaxConnections
parameter too low.I got it fixed in the end, and I expanded the documentation considerably. (49,000 words and counting in the wiki. Damn right I'm bragging!)
Putting off 'til next time, tempted though I am: reinstalling CentOS on the monitoring machine, which due to a mix of EPEL and Dag repos and operator error appears to be stuck in a corner, unable to upgrade without ripping out (say) Cacti. I moved the web server to a backup machine on Tuesday, and I'll be moving it back today; this is not the time to fiddle with the thing that's going to tell me I've moved everything back correctly.
(Incidentally, thanks to Matt for the rubber duck, who successfully talked me down off the roof when I was mulling this over. Man, that duck is so wise...)
Last day today. (Like, ever!) If I remember correctly I'm going to test the water leak detector...and I forget the rest; it's all in my daytimer and I'm too lazy to get up and look right now. Wish me luck.
And best of 2010 to all of you!
Irritating: chkconfig on RHEL/CentOS returns non-zero if a service
isn't configured for a runlevel. IOW, you can do:
chkconfig --level 3 foo
and have 0 returned if it's on, 1 if it's not.
But not SuSE; nope, it just returns 0 whether or not it's enabled, or
even if the service itself doesn't exist. Because, you know, grep
doesn't get used enough.
I'm doing this because I'm trying to use cfengine 2 to manage services. This works well in CentOS, where you can add something like:
service_foo_on = (ReturnsZero("/sbin/chkconfig --level 3 foo"))
and it'll work. ("servicefooon" is a bit of a misnomer, because I'm checking runlevels, not whether it's actually running.)
Update: Nope, I'm wrong. chkconfig --check does exactly what I
want. Many thanks to yaloki on #openSUSE-server for the help.
Install logwatch on Solaris fileserver.
Notice that logwatch emails are not coming in.
Log in and run logwatch by hand.
Inspect mail log and notice lack of any entries.
Notice that Postfix is in maintenance mode; start it up.
Notice continued lack of emails.
Notice that Postfix is running, which confused svcadm when told to start up Postfix. It fails to do so and fails to log this.
killall postfix, svcadm enable postfix.
man svcadm; svcadm clear postfix; svcadm enable postfix.
Run logwatch by hand; notice emailed report to "root@localhost.localdomain", which gets bounced by Postfix on the mail server because it's a non-existent host.
Resist temptation to go down that rabbit hole just now, and stick to the problem at hand.
Edit /opt/csw/etc/log.d/logwatch.conf and set MailTo to proper address.
Re-run logwatch and note that reports are still going to root@localhost.
After much swearing, notice that actually, logwatch is set to look in /opt/csw/etc/log.d/conf/logwatch.conf for configuration.
Edit that file, re-run logwatch.
Notice errors from Postfix: "postdrop[13848]: [ID 947731 mail.warning] warning: mailqueueenter: create file maildrop/908447.13848: Permission denied".
Run "postfix set-permissions". Test mail; still failing.
Check permissions on another system and set by hand.
Re-run logwatch. Still no email. Re-run with debug=high and get email.
Wonder idly about futility of self-aware log watching system that can't report on its own heisenbug-induced failure, crappy packaging practices, inability to check end-to-end email connectivity, other career options.
(Update) Realize that the emails show up if "Detail" is set to Medium or High ; Low, the default, makes the report silent.
(Update) Uninstall the package and reinstall, only to find that the symlink to conf/logwatch.conf is set up at installation, and that this is probably a case of $EDITOR breaking the symlink. Apply head to desk.
A few quick notes about building Fedora Directory Server RPMs for CentOS:
$instance_dir points to /etc/dirsrv, not /etc/fedora-ds.(Partly a memo to myself, and partly to help anyone in the same boat; edits have been disabled in the FDS wiki, so I can't add this right now.)
Heads up for those of you using Blastwave and CUPS: after upgrading to the latest stable version, printing stopped working for me (and a few users :-). I eventually tracked it down to the movement of two files: suddenly
/opt/csw/lib/cups/filter/pstopxl /opt/csw/lib/cups/filter/pstoraster
were moved to
/opt/csw/lib/cups/pstopxl /opt/csw/lib/cups/pstoraster
resulting in many error messages like Unsupported format text/plain! and Hint: is ESP ghostscript installed?. Moving them both back into place and restarting CUPS fixed things just fine.
According to Bacula (yay Bacula!) both files were in the right directory as of last night, and Blastwave's file list for Ghostscript shows the new location for these two files. A bug has been filed.
This is hilarious.
pkgsrc is still kicking my ass. The latest is a dupe of this bug; I can't tell right now if it's more weirdness with switching GCCs too soon, or something else.
OTOH, I came across MyReview today, and holy crap does it ever look like something my work could use. I've emailed the project thanking them profusely, and suggesting a Freshmeat page (am I the only one who turns there first when looking for Free software goodness?).
One of the things about pkgsrc is that it's very sensitive to paths and which compiler you use. (And fair enough; the whole process of bootstrapping a working set of tools for eight hundred thousand different OS' is ridiculous enough that it's a wonder it works at all. But I digress.)
Case in point: Solaris 10 machine today, installing pkgsrc on it for
the first time. I successfully compiled gcc34, added GCC_REQD=3.4
to mk.conf, and then went to compile kile. During compiling of
Mesalibs, one of its 3.2x10^6 dependencies, I got this error during
the final linking phase:
/opt/pkg/bin/libtool: ar: not found
Naturally it was there in my path, so WTF?
I eventually came across a message to the pkgsrc user's list
which suggested rebuilding libtool-base. This made a certain amount
of sense to me, as I'd built that package using the bootstrap (ie,
not-installed-from-pkgsrc) version of gcc to compile it; it was before
I figured out the GCC_REQD directive. So I ran:
$ pkg_delete libtool $ cd /opt/pkgsrc/devel/libtool $ bmake clean && bmake install $ cd /opt/pkgsrc/graphics/MesaLib $ bmake clean && bmake install
and everything was right again.
I've complained about Blastwave before, but this is just terrible.
Trying to install VLC on a Solaris 10 machine using Blastwave. Says
that CSWcommon is out of date, so please run pkg-get -u. As this
always includes thousands of prompts that look like this:
The following package is currently installed:
CSWoldapclient openldap_client - OpenLDAP client executables (oldapclient)
(sparc) 2.3.31,REV=2007.01.07
Do you want to remove this package? [y,n,?,q] y
## Removing installed package instance <CSWoldapclient>
## Verifying package <CSWoldapclient> dependencies in global zone
WARNING:
The <CSWoldap> package depends on the package currently
being removed.
Dependency checking failed.
Do you want to continue with the removal of this package [y,n,?,q]
...I look around for a way to automate this. And surprise, there
is, and I've missed it the whole time. My bad. So: pkg-get -f
upgrade it is, then.
It runs for 45 minutes and stops with an error about CSWcommon:
Current administration requires that a unique instance of the
<CSWcommon> package be created. However, the maximum number of
instances of the package which may be supported at one time on the
same system has already been met.
Hm, sez I. That's strange, but maybe that's what it's like for package
managers that suck. pkg-get -r common and pkg-get -i common, and
I'm ready for the upgrade again.
Somehow in the process I managed to remove the pkg_get package,
which (surprise) contains the pkg-get command. Fortunately I have a
backup copy around and use that to install pkg_get. Life continues.
And it's not for another 15 minutes after that that I notice that the package manager is going in loops. It keeps going over the same packages again and again, giving the same errror about unique instances each time. A quick search turns up this link, which tells me I'm a fool for believing the help offered by pkg-get:
$ pkg-get -h
pkg-get, by Philip Brown , phil@bolthole.com
(Internal SCCS code revision 3.6)
Originally from http://www.bolthole.com/solaris/pkg-get.html
pkg-get is used to install free software packages
pkg-get
Need one of 'install', 'upgrade', 'available','compare'
'-i|install' installs a package
'-u|upgrade' upgrades already installed packages if possible
'-a|available' lists the available packages in the catalog
'-c|compare' shows installed package versions vs available
'-l|list' shows installed packages by software name only
Optional modifiers:
'-d|download' just download the package, not install
'-D|describe' describe available packages, or search for one
'-U|updatecatalog' updates download site inventory
'-S|sync' Makes update mode sync to version on mirror site
'-f' dont ask any questions: force default pkgadd behaviour
Normally used with an override admin file
See /var/pkg-get/admin-fullauto
'-s ftp://site/dir' temporarily override site to get from
and that the correct way to do what I want is to run:
true | sudo pkg-get upgrade
I admit that I neither knew nor sought to find out what "default pkgadd behaviour" would be, so that's my fault. I admit that I was the one who borked things by removing the pkg-get command. I admit that I did not think to record all of this with script, so at the moment I'm going on scribbled notes and memory. This is not a bug report, which is what I really should be writing. These are all things I did wrong or badly.
But isn't this what apt has fixed? On its worst day, I've never
had to set up yes to be the drinking bird that would let me
get stuff done. And — when all was done, and I got to go back to
installing VLC — I've never had it depend on gcc.
Arghh. Arghh arghh arghh.
New emacs, woo! I've downloaded it and compiled it already, 'cos I am that l33+, thank you. But one thing: the tarball is signed by Chong Yidong, pgp/gpg key #BC40251C. I could not find any indication anywhere that this is the right key, or what the right key might be. A quick search turned up lots of posts on the Emacs mailing lists, bugzilla entries and such from him, so I presume it's okay…but it would be nice to make this explicit. (Even a search for the key number turned up nothing.)
This article about updating pkg-src makes me even happier I went with Debian. That is all.
Yesterday I got a new switch in at work. Good god, the 10/100 Procurves are getting cheap — $600 w/academic discount for a 2626. I was just going to rack it, but as always I couldn't stop once I got going; that server room needs a lot of cleaning up. Three hours later I emerged, bloody but triumphant: the network cables were cleaned up considerably, I'd identified the last of the mystery boxes (step-down transformer, not a UPS like I thought), and I'd figured out that the big UPS was only one-third loaded — plenty o' room. Once I get all the cleaning done, I'll post before-and-after pix, 'cos that will be one chunk of work I'll be damned proud of.
I think I'm going to have to end my experiment with Nexenta.
I've been running it for a couple months now on my desktop machine, and for the most part it does everything I'd want it to do. Sound doesn't work (built-in Intel chipset, 945 I think), but I haven't really looked into it too hard; the screen resolution keeps changing back to 1400x1200 for me in IceWM, but again I haven't really looked into it too hard. Firefox runs fine, xterms work, Emacs is there, and since it's a 2.8GHz P4 (cf. the 500MHz P3 I was running before), it's all ver' fast.
But when I started using it, I had visions of helping get it released; there are 90 bugs to knock down, and I could help with that. I can — I did (a little) — but with a 9-month old kid to help take care of, my time is et up pretty damn quick. A couple of hours on the weekend is the sum of my spare time right now, and that's for everything.
Why do I mention that? Because OpenSolaris needs a lot of learning, and Nexenta/GNU/Solaris needs a lot of work to get a beta release out the door. I thought I'd learn dtrace; I thought I'd knock down a half-dozen bugs while a growing community joined in.
That turns out not to be the case. And it's a damned shame, and I'm not helping matters any by giving up. I love the idea of Solaris + Debian. I'd like to see it up and running and grabbing people's attention and all the rest, but it's just not happening right now.
And so, in a few minutes I think I'm going to install Debian on here. It has what I want and lots more. There are plenty of people involved in the project, covering for my slacking. I'll be running testing, 'cos I've been doing it so long that it seems foolish to stop now :-). When I finally get around to upgrading the server that prompted this digression, I'll make it stable. I'll probably replace SuSE at work with stable, too.
And now for teh big finish...
$ pkg-get foo
Can't install foo; need newer version of libbar.
$ pkg-get libbar
No such package.
$ pkg-get -a |grep bar
Barlib The libraries of bar.
pkg-get -u foo does work. Supported? Who knows?(Edit: corrected name of Blastwave's package manager. So much for the moral high ground…:-)
Now that pkgsrc is working on the main Solaris box on work, I've been
trying to compile Kile, the lovely KDE-based LaTeX editor. KDE,
of course, brings in lots of other stuff; in other situations there
are probably ways of keeping things out (this is where Portage's USE
flags are nice), but as far as I could tell there was no way of
keeping, say, libogg out of the KDE build.
But the annoying part is when the build of KDE stuff failed because
OpenEXR, ILM's open-source high dynamic range graphics file
format, failed to compile. And why? Because hypotf isn't defined,
along with a bunch of similarly-named functions (atanf, cosf,
sinf and so on). I tried throwing -lm into LDFLAGS, but that did
nothing.
Some digging around in include files on a couple different machines turned up the problem: these functions were added in Solaris 10, and thus are not present on Solaris 9. I haven't been able to find any mention of this problem yet, at least for OpenEXR and/or pkgsrc; I'm hoping that there will be some other way of making this work.
I've finally got pkg-src working on the Solaris 9 machine at work. I've been banging my head against this stupid thing for two weeks, only to find that a) you really do need /usr/ccs/lib in your path, and b) you should check everywhere to make sure GNU's ld isn't in your path (if you're on Solaris). Now everything is compiling happily.
And in other news, I've been working on Nexenta for the last couple of weeks, trying to help knock down the bugs preventing a beta release. My development skills are limited but at least some of the problems are within my reach. There's not a lot of traffic on the mailing list, or on IRC, which worries me a little…but the commit log is nice and busy, so that's good. And I got that package of IceWM I was looking for. :-)
I installed RT at work a couple days ago using pkgsrc. This was the first time I'd ever used pkgsrc, and I have to say I'm impressed. Yes, it's just like a portable ports tree — but it's just like a portable ports tree, and I'm starting to think that's a very, very powerful idea.
RT went well except for the final install, where it complained and died. Fortunately, it turned out to be susceptible to exactly the sort of one-line patch that I have an affinity for. Not as cool as correcting Theo de Raadt's code, mind you :-) but still a good feeling.
Ah...RT, I've missed you.
Brian Cain told me to get my ass in gear and try out a Nexenta (I'm elaborating on his words a little) and I'm glad he did. I've installed it on my desktop machine on a second hard drive, and I have to say I'm impressed so far. Debian plus Solaris…damn, girl. Damn.
Everything is a .deb package, including all the SUNW stuff, and there appear to be a ton more packages available. Mutt and Emacs are there, as is procmail and fetchmail; I may see if I can get a package going for icewm, which'd be just about all I need. (Of course, ratpoison's already available…) (Update: someone's already working on it; haven't made it to the end of the thread yet, but it may already be done.) There was a lot of Gnome stuff installed that I don't want, but that's okay; Nexenta's deliberately emulating/duplicating Ubuntu, and anyway the install disk (which comes with Tetris, btw) has a minimal option which I suspect'd be right up my alley (for server or desktop).
I'm in the process of creating a zone right now, into which'll go
Apache2 and MySQL. I did trip over these bugs in the
process, but apt-get dist-upgrade fixed the first and some judicious
editing of /usr/lib/nexenta-zones/elatte-unstable.bootstrap fixed
the second (I'm guessing they haven't made a new package since the
fix). (Update: My own damn fault for not noticing that the new
version was in unstable, not testing. I'm upgrading now.)
/export/home was set up with ZFS, and I've made a snapshot
already. The GRUB menu entry was not correct — it pointed at the
primary IDE drive (hd0) instead of the second (hd1) — but again, that
was easily fixed. I should file a bug on that.
I still have some questions. I'd like to know (and will try to watch to find out) how often they update their packages, especially security fixes. I'm curious to see how closely they follow OpenSolaris.org development…though since I only have a hazy idea how OS.org do it, I'm not really sure what to look for. And of course, this is an unstable distro; I might want to hold off on replacing the server with it.
But for desktop use and/or experimentation, this is neat stuff. I can always get my mail on my firewall if need be. :-)
People have been calling me out on my last post, and that's good; I love a good argument^Wdebate, and doubly so when it comes from people w/more experience than me. So I'm going to start responding to the comments, laying out where I'm wrong and where I still think I'm right.
I said:
OpenSolaris: If I wanted to upgrade everything by hand, I'd stick with Slackware.
Bzzt! As I found on on a recent episode of BSDTalk, NetBSD's pkgsrc is available for over nine hundred thousand operating systems, including Solaris and Slackware Linux. Tha's right, both premises in that statement were wrong.
Not only that, pkgsrc can be tucked out of the way so that it doesn't interfere with the rest of the system…so I could even throw it on Thornhill right now, Slackware and all, and start using it instead of my own half-assed build script for Apache/SSH/PHP/OpenSSL/mod_ssl (which, in my own defence, works pretty darned well).
In fact, tomorrow I'm heading out to The Other University to set up two new X4200 servers, and I'm seriously considering adding pkgsrc to them — if only to avoid having to compile (and botch) Lapack and Blas. If that goes well, I may start adding it to the main server here so that we can easily get more up-to-date versions of Firefox et al. (Though I could probably get them from Blastwave…this has been a low enough priority for me so far that I haven't really looked into all my options.)
That is not to say it's perfect:
It is possible, and in the case of updating a package with
hundreds of dependencies, arguably even likely that the process will
fail at some point. One can fix problems and resume the update by
typing make update in the original directory, but the system can have
unusuable packages for a prolonged period of time. Thus, many people
find 'make update' too dangerous, particularly for something like glib
on a system using gnome.
To use binary packages if available with "make update", use
"UPDATE_TARGET=bin-install". If package tarball is not available in
${PACKAGES} locally or at URLs (defined with BINPKG_SITES), it will
build a package from source.
To enable manual rollback one can keep binary packages. One method is
to always use 'make package', and to have "DEPENDS_TARGET=package" in
/etc/mk.conf. Another is to use pkg_tarup to save packages before
starting.From the Swedish NetBSD wiki.
It's nice that manual rollback is doable; that's always my big paranoia when it comes to source-based upgrades.
That last complaint is not as fair as it could be. I mean, I'm not going to be upgrading Gnome on either Thornhill or the two new Sun machines. And at around 80 packages, it would be damned difficult to try and recompile it all without starting with a clean slate. But this sort of nonsense with Gnome is what put me off the ports tree in FreeBSD.
(I was going to put in something about how Debian doesn't need that sort of thing, but I should research that first.)
libpst is a command-line tool that converts Outlook .pst files into standard mbox files, the way T&R intended. Wish I'd known about this before…
One of the outstanding feature requests is listing and extracting individual messages. Maybe I'll take a look at this.
In other news, I borked my home machine (Debian testing) by trying to extend a partition w/ReiserFS. That gave me a perfect excuse to upgrade to a bigger disk and reinstall Debian.
Next up is maybe looking at replacing my venerable copy of Slackware 9 with a Debian install, too; the ease of installing and upgrading Debian packages is just too good to pass up.
I did consider other OSs:
And yes, I realize I'm damned ignorant, and that a server should not be exciting. But I'm convinced that a big part of running a server successfully is ease of upgrading, whether security fixes or new app versions, and Debian is just wonderful.
Solved a ghostscript problem at work yesterday; not a big deal in itself, but I'd always had this impression that GS crashes were dark, nasty, impenetrable things that I could not possibly understand. I mean, c'mahn, look at this error:
$ ps2pdf report06w5060.ps
Error: /invalidfont in findfont
Operand stack:
Fi 87 --nostringval-- 55 45 --nostringval-- 65 74
74 111 74 83 46 65 65 83 83 83 83 120 46 2
--nostringval-- 4
6 83 83 46 74 83 74 83 83 12 --nostringval-- 92
83 101 1 --nostringval-- 101 120 1 --nostringval-- 138
4 --nostring
val-- 120 120 101 101 120 111 101 101 19
--nostringval-- 55 42 1 --nostringval-- 83 2
--nostringval-- 55 35 --nostringv
al-- 83 83 2 --nostringval-- --nostringval-- 45 166.044
Times-Italic Font Times-Italic 496086 Times-Italic
--nostringval-- Times-It
alic NimbusRomNo9L-ReguItal (NimbusRomNo9L-ReguItal)
NimbusRomNo9L-ReguItal (NimbusRomNo9L-ReguItal)
NimbusRomNo9L-ReguItal
Execution stack:
%interp_exit .runexec2 --nostringval-- --nostringval--
--nostringval-- 2 %stopped_push --nostringval--
--nostringval-- --nostringval-- f
alse 1 %stopped_push 1 3 %oparray_pop 1 3 %oparray_pop
1 3 %oparray_pop 1 3 %oparray_pop .runexec2
--nostringval-- --nostring
val-- --nostringval-- 2 %stopped_push --nostringval--
--nostringval-- 74 4 %oparray_pop 75 4 %oparray_pop
--nostringval-- --nostringv
al-- --nostringval-- --nostringval-- --nostringval-- false 1
%stopped_push 78 5 %oparray_pop --nostringval--
--nostringval-- --nostring
val-- 5 -1 1 --nostringval-- %for_neg_int_continue
--nostringval-- --nostringval--
Dictionary stack:
--dict:1046/1123(ro)(G)-- --dict:0/20(G)-- --dict:75/200(L)--
--dict:103/300(L)-- --dict:17/17(ro)(G)--
--dict:1046/1123(ro)(G)--
Current allocation mode is local
Last OS error: 2
Current file position is 95763
AFPL Ghostscript 8.00: Unrecoverable error, exit code 1
Then, in desperation, I JFGI and found the problem: for some
reason, the fonts had disappeared. This is an old install with lots of
overlapping installs of everything, so it's hard to tell why it
might've happened. However, it should just be a matter of either
getting rid of the old install (rm /opt/bin/gs* (and yes, I know
that's bogus)) or setting GS_FONTPATH and GS_LIB
appropriately. (Or figuring out why they got borked…hm.)
OTOH, on the same machine I've got The Case Of The Missing Java:
$ java
There was an error trying to initialize the HPI library.
Please check your installation, HotSpot does not work correctly
when installed in the JDK 1.2 Solaris Production Release, or
with any JDK 1.1.x release.
Could not create the Java virtual machine.
instead of (same version of Solaris, too):
$ java
Usage: java [-options] class [args...] (to execute a class)
or java -jar [-options] jarfile [args...] (to execute a jar file)
which kind of worries me since its, like, Solaris and all, and java really should be working. Sigh.
A real OS shows messages when it's booting to let you know how far along it's gone. If something goes wrong, you can see where. A real OS doesn't have a fucking marquee of slowly-changing colour that you have to stare at intently from six inches away to see if it's frozen in place and yet another cold reboot needs to happen. Oh, and it logs how the boot went. Every time. Without being asked.
Also, if an OS comes with a package manager -- and a real OS does -- it shows you the fucking version of the software that's installed. A real OS knows that knowing ~FooTastic installed is only half the battle -- knowing that it's the buggy 6.2 build is just as important.
I try to give Windows and Microsoft a chance, I really do. But this is fucking ridiculous.