I'm honestly curious. Is a large portion of the Hacker News audience just less operationally experienced compared to the programming side of things? It seems like Linux 101 lists like this make it to the top of HN pretty regularly but I'd equate this to a post about Python that listed for/while loops, if statements, how to create a class, etc. Am I blinded by my own experiences or do very basic Linux seem to do very well on this site.
In my experience, systems admins tend to be the most familiar and comfortable on the shell, though even there, mileage varies markedly. Devs can vary a lot, I find skills rare among UI / design types, even those (most nowadays) using Macs, and for administrative / project management types, it's often very little. Based largely on casual observation, mind.
The other point is that specificity of knowledge can very greatly, and IT is all about specialization. Even people who are good at multiple tools often have their limits, and I'll see specialization in development, databases, networking, build/release management, QA, systems development (as opposed to app or UI work), and related specialties such as networking and others. Specific skillsets too are often accidents of opportunity and history -- having moved through a specific shop or acquired a particular skill is very much dependent on where you've been, much less a mark of innate ability.
One recent team I worked with had access to a set of highly complementary talents: very strong systems knowledge, database, overall app and enterprise architecture, and a networking consultant. Resolving a number of issues and enhancements turned out to rely very much on each individual's strengths, where what any given member could provide by way of easily accomplished tasks (seconds or less) would have set the rest of the team hours or days.
What I find particularly damaging is environments in which expertise is guarded while at the same time begrudging others for what they didn't know (and not providing for knowledge transfer) generally leads to negative engagements. This is almost always present in part (nobody knows everything or has time to teach everyone), but in particular cases has been a very strong undercurrent.
I feel like OS X has a big hand in this discussion. Most developers I know now use OS X and only touch it's Unix underpinnings when required to do so. They don't use SSH so much, they'll use sshfs or some other remote file system setup, they'll use a git GUI in their IDE and a (S)FTP client more often than not. The dev community over the past few years seems to have embraced nix styling in their projects more and more so I think en mass people are getting more exposure to the CLI and therefore, getting more curious about what else it can do, leading to another blog post about nix commands, how to use them, etc.
The irony is I remember when people gave me shit for running OS X back in 2001 and being "stupid for not running windows" and "making my life harder". The schadenfreude of a Windows user watching me when something didn't just work automatically or I had to boot up VirtualPC back then was far from subtle and peppered with reminders I could "just run windows dude". Now days, it's still there, but it's much more common and much stronger coming from a Mac user than any Windows users I've encountered as of late.
I'm willing to bet there isn't but a handful of people in this community that "knows everything on that list", remembering everything on that list is... well, unneeded?
9/10 times when I'm using a command that isn't part of my "daily list" I'll need to consult man pages and/or Google.
I've used xargs hundreds of times, but depending on when I used it last, I can never remember if I need -0 or not, and is it $1 or {} or wait, no no, I'm using awk, so it's $1, and I need to changes something... is it needle, haystack, replacement or... hell, I'm not even sure if it's substr or just sub... errrr.
Not to mention some of these are distro specific, in some you use service name start, in others it's /etc/init.d/service start. Some it's adduser, others it's useradd, etc. It's relative to the distro a lot of the times... are the files in /usr/opt/ or /opt/... or maybe they're in /usr/local/opt/... Also they list stuff like locate, but not updatedb. Not a single mention of dd or shred or /etc/resolv.conf or netcat (I fucking love netcat!) or unzip (far more likely to need to unzip an archive than add a user in your day to day life).
We're not command libraries, we're duct tape mechanics.
I don't consider myself in any elite minority for it, but I feel I have strong familiarity with everything on that list. While maybe a bit of curating might be necessary, _every_ developer whose code comes anywhere near a Unix descendant should be just as familiar to be considered anything above 'junior'. Most teenagers are perfectly capable of this understanding.
I find my job wearing a DevOps hat to be very often much more difficult than it ought to be because so many programmers clearly don't understand the OS layer they're running on (both when supporting internal developers at various places which I've called $work, and using OSS and commercial software in that role)
There's no shame in running `man xargs` to get the syntax just right (we're not dictionaries) but general knowledge of what those tools are and how to use them are _essential_ to being a good developer... this is an essence too often neglected by universities and professionals.
I agree completely. My point was when I looked at the list the first thing that jumped out at me was dig... yeah, I know what dig is, what it's used for, but I just hadn't seen it in a while... that's how often I actually have to use it. There isn't a reason for me to really keep it in my mental dictionary of known commands because of it's overall utility to me. Likewise commiting logrotate flags to memory, pretty much useless, how often do you honestly have to run that by hand or use it in a day in day out situation. If you answer "often" to that, then we'll need to get you up to speed on more core components like crons and shell scripts.
>There's no shame in running `man xargs` to get the syntax just right (we're not dictionaries) but general knowledge of what those tools are and how to use them are _essential_ to being a good developer... this is an essence too often neglected by universities and professionals.
I don't really think so. You can get by fine developing a lot of things without knowing linux tools. They are helpful, though. What if you're working on Windows like a ton of devs do?
It's always good to have a diverse set of skills, but I meant (and said somewhere) the things I said for those whose code ever runs on nix. I also don't just mean a mastery of the commandline tools but also how various pieces of a nix system run (most recently my irritation has come from bad logging) and general Unix philosophy.
So is it maybe that Python is a language with much smaller surface area and (most likely) much more consistent design, so we simply don't need a cheat sheet like this? By rough estimate, my Mac came with over 1000 commands - no structure, namespacing.
I love that xkcd. I really hope someone will take up the task of replacing the UI of *nix with something better. For me, it's like Windows 8: it was a nice idea to combine touch and traditional user interface - it just turns out to be inferior to either being done well. Writing shell scripts is just, let's face it, horrible, and the command line interface far from ideal. It works, it has for decades. But it's not like there has been a lot of competition to compare with.
Sibling is complaining about academia and universities - but again, apart from sys admins, who don't teach, none of the academics know this stuff. Learning and teaching Python (Java) is in some respect easier but more importantly interesting - it just doesn't make sense to learn all these commands without actually needing them (whoever comes up with an exercise to use them all together gets a medal).
I agree with your sentiment but if you find the CLI to be a lack luster interface the only thing I can think is that you've not spent enough time using it correctly. I was a Windows guy, then an OS X guy, then a Linux guy. I found that I preferred CLI more than GUI's so I moved to Linux full time when I started getting frustrated with GUI tools and Darwin ports and the like.
Now days it annoys me if a library doesn't have a CLI. Even on my personal machines I run the least amount of GUI possible for resource and usability purposes. I don't want a new GUI or a better CLI (zsh, csh, bash, ksh, etc. do just fine). I just want tools that utilize it properly to support it's most powerful functionality (pipes, output redirect, proper exit codes, etc.)
A lack of familiarity with it is no institutes fault, it's a personal fault, if you find your self looking at a CLI interface, and then resorting to searching Google for a GUI wrapper for that interface, the ignorance isn't due to a failure of the CLI, it's a failure of the user being intimidated or ignorant.
Also I have no issues writing shell scripts, I use them all the time to save me tons of work. If you're a programmer then you really get to know your system and you really get to customize it to your liking.
An example would be switching to i3 from Gnome/KDE. I was a long time KDE/Gnome/XFCE user and my laptops track pad settings would never stick after a reboot when I used the GUI tools for disabling the touchpad tap to click or just disabling it in general (thinkpad nipples ftw). After moving to i3 and really getting more access to the core of the system I ended writing a couple little shell scripts for enabling disabling my touchpad for when I need/want it. What used to be a 5 step operation (open gui > click tab > uncheck box > click save) has now become (super+space, type disa[tab](autocomplete disable.touchpad), enter). I have a second script to enable it again, the contents of which are small.
I agree that CLIs are ofter better than GUIs, but I think the point was that the Linux CLI tools could have been designed better. For example, personally I don't like how gzip/gunzip comes in a pair, but there's no "untar" command, instead you must pass the -x argument to tar.
I'd disagree with the assertion in your first paragraph. The Unix CLI toolset has evolved in a very similar way to any other software ecosystem. To compare with Python, the GNU coreutils/binutils and friends could be thought of as analagous to the Python core with other toolsets fulfilling a role similar to packages from PyPI. The standards and interface issues for both ecosystems are similar but luckily Python has more leeway to alter compatibility (ref changes in 3k).
There are all sorts of issues with altering the nix toolset, and we mustn't forget the importance of POSIX for compatibility. We can't simply decide to 'replace the UI of *nix' without fundamentally changing how the nix ecosystem and community operate (and I wouldn't want to). OS development is not agile and rarely compat breaking, and for good reason.
I also read the sibling as promoting a fundamental understanding of the underlying platform being developed on, not so much pushing teaching of just CLI usage. Knowing how, e.g., the filesystem works and the caveats involved is something that can lead to much better software design decisions. And knowing how to chain a cheap sed instead of just throwing Python and the re module at a glob.glob listing can be just as useful.
a seperate issue : if you're going to make lists like this, please don't include distribution specific commands (like yum or apt-get) on 'linux' cheatsheets. they belong on redhat or ubuntu cheatsheets, respectively.
Similarly irritating : the use of distribution specific package names.
A good list will mention the alternatives (even if it doesn't explain all of them), but still include them.
There are a lot of tasks which one ought to be familiar with which aren't anywhere near cross platform, and keeping your list to very widely available tools will leave huge gaps in your knowledge and makes for rather short lists.
While I find your question specious, there were about four commands I wasn't immediately familiar with there (some are somewhat distro / choice specific with other alternatives being present), and that's with 17 years professional experience. I'd run across the vast majority within my first year or two (and had been using others for years on Unix systems).
That said: some of the commands I use most often are exceptionally basic. And in the case of others, I'm learning things all the time. Say, such as mplayer's constant-pitch video playback speed argument:
Most developers I know do not have a solid grasp of command line usage, regardless of platform or age. It is at least as painful watching them use a shell as it is watching the stereotypical grandma use a computer in general.
I think that is the case. Even though the name is Hacker News, I suspect that most visitors are interested in the startup/business side of things, even though there is an important core of technical people.
I find it odd that the single most important command is absent from the list: man (w/ -k, or apropos). The article could really do with a section 0 to explain it.
Armed with the ability to actually find commands and RTFM, you don't need to try and remember stupid lists of commands.
Obviously it's helpful to remember commands, particularly frequently used ones - but you should start with the right mentality: that of being able to use your initiative to work something out, rather than resorting to google and not having to think.
I'm curious as to why the sheet recommends 'ps aux | grep' without bothering to mention the self-match (and tricks to avoid it like doing grep '[f]oo') instead of just presenting pgrep with the -f flag.
This reads like a naive user primer rather than a particularly detailed cheat sheet. I'd highly recommend that anyone interested in improving their nix-fu beyond what is presented in the article look at something like the Unix toolbox[1] (which I personally still find useful and recommend frequently) and take the time to dig through the man pages in order to learn how to use and chain these tools.
IMO "ps aux | grep" is better than pgrep, it's nicely formated and allow option like "--color", which allow to quickly find the option/process you were looking for.
One of the most confusing yet important thing I learned as student from working with Linux is the execute bit on directory. It is important not to do chmod -R 755 mydir because you will give execution bit to files as well.
GNU chmod lets you say *chmod -R a+X mydir" (an uppercase X) which, instead of giving everybody execute permission on all files and directories, will give everybody execute permission only on directories, and files that already have some execute bit set.
[edit: If I'm reading this right (I seldom have reason to look up Posix things, being generally concerned with concrete implementations) -- it's actuallly a BSDism that's been incorporated into Posix:
The first draft of my comment started "GNU chmod, at least,..." but then I wound up with too many comma-delimited clauses and I dropped the qualifier to keep things readable. Thanks for looking up citations. :)
Hm, no "lsb_release -a" under system? Also, I've come to rely more on ls -l /proc/${some_pid}/fd than lsof -- but if you only have to choose one, I suppose lsof is better.
I couldn't immediately tell from your reply whether you didn't know about pushd/popd or were mentioning them as something missing from the list. Since I don't seem to see them in the posted list I'm going to assume the latter.
For anyone who is curious about pushd/popd, pushd will change your current working directory and adds your new working directory to the top of a stack. popd will remove the top directory from the stack and change your current working directory to the new top of stack. This is helpful in scripts where you need to change directories for some action and then back out to where you were. You can also use the dirs command to list the stack.
One reason I could see omitting these from the list is that they are shell builtins rather than external commands as the vast majority of the list is.
Yeah, whenever you cd, OLDPWD gets set to the directory you just changed from. cd will substitute $OLDPWD when you cd to "-". Perhaps even less well know, the shell itself will expand ~- to $OLDPWD so if you ever want to see where cd - would take you, or just list the files from your previous working directory you could do "echo ~-" or "ls ~-" respectively.
The new one for me today was "ditto," which I immediately googled to see if there was a linux equivalent (there is, and it seems to be 'cp -Rp --parents src/ dest/').
Edit: Compared to the OP link I like that it doesn't truncate words arbitrarily to wrap lines :)
