The Pragmatic Programmer: From Journeyman to Master

by Andrew Hunt and David Thomas

The “Pragmatic Programmer” is a high level overview of a wide range of practical topics in software development. It covers just about every aspect of software development including working in teams, source control, writing bullet-proof code, thinking critically, remembering the big picture, the benefits of knowing your tools, effective communication…and the list goes on.

“The Pragmatic Programmer” is organized around a series of practical tips for becoming a better software developer. Many of the tips are immediately implementable, and each tip is explained in a logical way and supported by entertaining anecdotes and quotes. The book contains a handy tear-out card that lists all of the tips.

This book is recommended for anyone who wants to be a better coder. It is probably best suited to beginner and intermediate developers, but advanced developers may find it a fun and worthwhile read as well.

Also recommended by Florian Potschka and Jason.

The Art of Computer Programming, Volumes 1-3 Boxed Set

by Donald E. Knuth

According to The New Hacker’s Dictionary, a safe answer when you don’t know something is always: “I think you can find that in Knuth.” The Art of Computer Programming (TAOCP) is a probably the definitive computer science text. This boxed set contains the first 3 volumes of a planned 7 volume set. Volume 4 is unfinished, and is currently being published as a set of smaller books called fascicles. The 3 volumes included here are: “Fundamental Algorithms,” “Seminumerical Algorithms,” and “Sorting and Searching.”

I’m a bit embarrassed to say that I haven’t read TAOCP - I’ve always found it a bit hard to justify spending $190 on a set of books, even if they are masterpieces. (Update: Amazon currently has the set marked down to $153.) I would recommend TAOCP to anyone who wants to fill in any gaps in their computer science knowledge, and who has a lot of pocket change laying around.

Also recommended by seb and Dennis Grooves.

The Art of UNIX Programming

by Eric S. Raymond

This book does a great job of explaining the old “hacker ethic” and explaining why *nix is the way it is. “The Art of UNIX Programming,” contains case studies and commentary from many of the big names in the *nix culture: Ken Thompson, Steve Johnson, Brian Kernighan, David Korn, and others.

This book really covers a lot of ground. The first part of the book is an introduction to the *nix history, culture, and philosophy. The main points brought up in the introduction are then well-illustrated by case studies. The book goes on to talk about the various tools that are available in the *nix ecosystem, and how to be a successful participant in the FOSS community.

This book is recommended for anyone who has an interest in operating systems history, the open source culture and philosophy, or just learning about different tools and techniques that will help in becoming a better coder.

Also recommended by jason and Tet.

The Little Schemer - 4th Edition

by Daniel P. Friedman and Matthias Felleisen

“The Little Schemer” (editions 1-3 were called the “Little Lisper”) uses a unique question and answer technique to create a dialog that really draws you into the book. It starts from first principles and gradually works up to more advanced topics like the halting problem and writing a Scheme interpreter.

Despite the title, this book isn’t really about Scheme; it covers only the basic features of the language, and doesn’t even introduce local variables. However, a more thorough treatment of Scheme is given in the next book in the serious, “The Seasoned Schemer.” What “The Little Schemer,” does do well is teach recursion and the art of “thinking in Lisp.” Recursion is a recurring theme that is continually expanded upon throughout the book.

This book is recommended for anyone looking for a mind-expanding treatment of recursion and the art of thinking like a lisper.

Also recommended by landon dyer.

Peopleware: Productive Projects and Teams (Second Edition)

by Tom DeMarco and Timothy Lister

“Peopleware” focuses on the sociological aspects of software development. The authors state that for the overwhelming majority of failed software projects, the failure was due to sociological, not technical, problems.

This book is very well regarded in the industry, and I think a large reason for that is the underlying theme of Peopleware: developers, when given the chance, will do the right thing. Furthering this idea, a manger’s job is not to make people work, but to give people the ability to work. This means (among other things) private offices with closing doors for developers, and coders who are given intellectual responsibility over their jobs. The points are backed up by statistical evidence and often humorous anecdotes.

This book is recommended for anyone who manages software development projects, or aspires to lead a team of developers. Buying one for management could just convince them to ditch the cubicle idea.

Also recommended by Florian Potschka.

Snow Crash

by Neal Stephenson

“Snow Crash” is the obligatory science fiction book in the list. This book is one of the defining books in the cyberpunk genre, right along with Bruce Sterling’s work. “Snow Crash” is the story of Hiro Protagonist, last of the freelance hackers and the greatest sword fighter in the world, who delivers pizzas for a Mafia-controlled pizza franchise. He teams up with Y.T., a spunky young female skateboard courier in order to help his friend Da5id who was the victim of a computer virus with biological effects.

This book ties together ancient Sumerian myths, post-modern computer viruses, and biological hacking. The result is an almost matrix-esque hacker world view. For the most part, the book moves along very quickly from one surreal action sequence to another. My only complaint was that the section on Sumerian culture really bogged down the pace of the book.

“Snow Crash” is a must read for anyone who enjoys the cyberpunk genre and/or fast-paced, action-packed fiction.

Also recommended by Jamie.

Stand out from the crowd.

Very few programmers have portfolios. Judging by the candidates I have interviewed, I would guess less than 5% take the time to put together a portfolio of their work. So just having a portfolio in the first place can put you ahead of a large number of potential candidates when looking for a job.

Using your portfolio during an interview provides the interviewer with visual cues that bring your work to life. Showing not just telling is a powerful thing. It makes your achievements concrete and memorable, which makes you yourself memorable.

What to include?

Obviously, you only want to include your best work - this is a chance to showcase your talents. If you have experience on a lot of projects, you have the luxury of choosing projects based on the image you want to project. Do you want to show that you have a wide breadth of knowledge? Or do you want to leave the impression that you are an expert with a specific language or technology stack? Chose the work to include based on how you want an interviewer to see you.

After narrowing down the project list, include relevant details for each project. Basic details that should always be included are the project name, your role in the development cycle, a description of what the code does, a high-level description of how it works, and the skills, programming language, and technology stack used.

Think about any other optional information that should be included. If any of the work was open source, you could include source code examples. Provide the name of the client / company the work was for, especially if they are well-known. Including lines of code or number of source control check-ins can provide some measure of the size of your contribution to the project. Consider taking a page from the book of our graphic designer cousins and include relevant images. Or write about any challenges that were overcome during the course of development. There are quite a few optional elements that can be included, but I would suggest choosing a couple things that give the most insight into the projects and yourself so that the portfolio stays succinct.

Showing the portfolio.

The time to show your portfolio is not during the initial contact with the company. A potential interviewer isn’t likely to look at a portfolio before you pass a resume screen. I know I have trashed a few resumes that tried to be both resume and portfolio at the same time (it’s very hard to do both well at the same time). That said, if your programming portfolio is online, it probably won’t hurt to include a link to it on your resume.

The interview is where the portfolio will really pay off. Bring plenty of copies, and make sure that everything is well organized so that you can quickly point an interviewer to specific examples when they relate to interview questions. This gives you the opportunity to show what you have done, not just talk about it. Use the portfolio as a talking point and visual aid. I tend to get mildly panicked during interviews, which can make me forgetful; a portfolio gives a quick overview of my important work that I can use to jog my memory and keep myself focused.

Portfolio Examples

• Lester Chan’s Portfolio
• White Night Logic’s Portfolio
• My Programming Portfolio

Leave a link to your own portfolio in the comments if you think it would make a good example, or if you would like it critiqued. And of course feel free to critique the examples above (including my own).

A note on the definition of famous

So what makes a programmer famous? Inclusion on Wikipedia’s page of famous programmers, with the added requirement that famous programmers have their own Wikipedia page. This gets rid of a few programmers who were added for obvious self-promotion - I’m looking at you Blazie Engineering guys, whoever you are. Unfortunately, it also takes a few great programmers out of the running, like Brian Harvey, creator of UCB Logo and Steve Chamberlain of cygwin fame.

By way of a disclaimer, I made one edit to the page while compiling the data. Carl Sassenrath was credited for operating systems and algorithms, and while Carl is a great guy (and was my boss for a short while) who has made some great contributions to those fields, I didn’t think it right to credit one person for two entire areas of computer science. So that leaves 222 programmers who fit the definition of famous, and 400 projects that they worked on.

Famous Projects

Looking at the first graph, if you are looking to get famous, invent a language. Just over 23% of projects that famous programmers are known for are either inventing a language and/or writing a compiler. This group of famous programmers includes notables such as James Gosling (Java), Alan Kay (SmallTalk), and Guido van Rossum (Python’s Benevolent Dictator for Life).

Writing a game, writing an OS, and founding a company or organization are also activities worthy of fame and notoriety. A small, but substantial number of programmers made their name creating web browsers, authoring books, inventing algorithms, writing command line tools, working on new hardware, creating text editors, creating document markup or formatting languages, inventing protocols, writing databases, or in the office software realm.

Interestingly, 31.49% of projects couldn’t be put into a category that had more than 5 members. This is a huge long tail, and includes everything from coining the phrase “computer virus” (Leonard Adleman), to Beowulf clustering (Donald Becker), to being a coder on the IBM PC project team who wrote the Control-Alt-Delete keyboard handler (David Bradley). Another highlight from the ‘other’ category is Charles Simonyi, inventor of the dreaded Hungarian notation, and to throw in a bit of random trivia, also the fifth space tourist.

How many projects does it take to become a famous programmer?

Not many. 52.36% of famous programmers are known for their work on only one project. Obviously these guys have been involved in a variety of projects over the course of their careers, but for the majority, just one project propelled them to fame. On the flip side of the coin, there are a few people who achieved fame through their work on a number of smaller undertakings. For example, Lou Montulli is credited with creating the Lynx browser, cookies, the blink tag (ugh what were you thinking!), server push and client pull, HTTP proxying, HTTP over SSL, browser integration with animated GIFs, and being a founding member of the HTML working group at W3C.

What happened to the women?

There are few women in the ranks of computer programmers, and pitifully few that can be called famous. So few in fact, that transsexuals are giving women a run for their money. The dataset includes 215.5 men, 5.5 women and 1 transsexual. More on that .5 of a person shortly.

Famous women programmers are Adele Goldberg, who worked at Xerox PARC laboratory and wrote a number of SmallTalk books, Grace Hopper, a pioneer in the field who wrote the first compiler, Ada Lovelace, credited as being the first programmer, Emily Short, who played a major role in the development of the interactive fiction development system Inform 7, and Pamela Crossely, creator of SIMPLE for academic management of web pages and related Unicode-capable applications for teaching and research.

Also of note is Roberta Williams, who was only able to credit the women’s side with half an entry since she shares her notoriety and Wikipedia entry with her husband Ken. This is the where the .5 comes from in the men’s and women’s datasets - together the husband and wife team counts as a full person. They are credited with founding Sierra On-Line and writing and designing several games, including the popular King’s Quest series. Their story is partially chronicled in Hackers: Heroes of the Computer Revolution.

Danielle Bunten Barry is the lone transsexual in the list. She is known for the M.U.L.E. multiplayer video game written in 1983.

So there it is. The most famous and revered programmers in our culture work on a variety of projects, especially compilers. They are well-known for only a small number projects, and the overwhelmingly vast majority of them are male.

How would you understand memory management and pointers in Java? How would you learn about data structures in assembly? You can’t. Thats how. Java hides the details of memory management and pointers so you don’t have to deal with it, but knowing what is going on under the covers will make you a better programmer. Assembly is so low level that if you use it as a vehicle for implementing a B-tree you will loose most of the big ideas about the data structure and algorithms in a sea of register manipulation and instruction set details. This list is for people who want to get at all the big ideas in computer science, which necessitates learning a wide breadth of languages. Its strongly in the language-as-a-vehicle camp, not the language-as-a-career camp.

Assembly

Assembly will get you right down to the bare metal, working with registers and architecture-specific instruction sets. In short, assembler requires learning the fundamental architectural principles of modern computing equipment, which is essential for writing better higher level code that takes advantage of bare metal. It will also give you a fine appreciation for the dangers of writing spaghetti code.

Usage of assembly today is fairly limited, but encompasses many interesting areas such as reverse engineering, embedded systems, viruses, real-time systems, and compilers. There are a variety of assembly languages out there. MIPs is a good choice for its simple instruction set, and x86 is a good choice for its ubiquity.

Big ideas: bare metal, compilers, optimization, self-modifying code

C

C is the direct ancestor of some of the most widely used modern languages: C++, Java, C#, and PHP just to name a few. Knowing any one of these languages will allow you to pick up any of the others easily, and to communicate ideas with developers from a variety of backgrounds. In short, C is the lingua franca. Learning C has the added benefit of introducing memory management and pointers - things that higher languages like Java hide with automatic garbage collection and object references.

C is used all over the place: in every modern non-toy OS, Java VMs, the Ruby interpreter, and the Apache web server.

Big ideas: Memory management, pointers, complexity analysis

A Lisp Dialect

Lisp will make you think differently. I can think of no better way to put it than that. The language is simply structured in such a way as to make both problems and their solutions look differently. The structure of the languages begs the usage of recursion, maps, and closures - things that aren’t seen nearly as often in other languages.

Lisp is worth learning for… the profound enlightenment experience you will have when you finally get it; that experience will make you a better programmer for the rest of your days, even if you never actually use Lisp itself a lot. - Eric Raymond, How to Become a Hacker

Lisp dialects are used heavily in artificial intelligence, and as scripting languages within other applications such as Emacs and AutoCAD. Scheme is a good dialect to learn since it is the language used in SICP - one of the best computer science texts of all time. Or learn Emacs Lisp for a foray into dynamic scoping.

Big ideas: recursion, closures, artificial intelligence

A Scripting Language

In an era where we have hackable Coffee Makers and everything is online, scripting languages are the glue that binds the world together. These higher level languages can be a way of really strutting your stuff and a platform for putting into practice many of the big ideas in computer science - regular expressions, concurrency, complexity analysis, scalability, architecture, data structures, databases, and so on.

Scripting languages are commonly used for lightweight web applications and are often a good substitute for shell scripting. Pick up one of the P’s in the LAMP stack (Perl, PHP, Python) or Ruby.

Big ideas: complexity analysis, scalability, architecture, data structures, databases, etc.

The above four categories of languages have enough breadth to get a taste for the vast majority of the big concepts in computer science, after which it is much easier to absorb select topics in-depth, write solid code in any language, and pick up the unique paradigms of any new language you learn.

What programming languages would you put in your own essential toolbox, and why?

Image attribution: kimberly hurst

WikiViz Graph Renderer

by Chris Harrison

WikiViz is a Java program that renders graphical representations of graphs containing nodes and edges. In this case, the nodes are Wikipedia category pages and the edges are links between pages. The algorithm works by first placing each node, and then iteratively moving strongly linked nodes closer together and springing weakly linked nodes farther apart. Overlapping nodes are nudged away from each other by a different proccess. WikiViz can be used on other datasets such as related books in Amazon.

by Shantanu Goel

Shantanu walks through an installation of motion, a FOSS software motion detector program for Linux. Motion comes with a small web server and can be configured to publish images to the web whenever motion is detected. It also has the capability to draw a handy box around whatever part of the frame moved. The tutorial provides a choice of 3 different commands that will send you a Twitter update whenever motion is detected: curl, wget, or a custom Perl script. Catch burglars. Or just see what your cats do all day.

by Simon Jansen (ASCII Artwork) and blinkenlights.nl (Telnet availability)

Simon wrote the asciimation, which an animation created by displaying frames of ASCII art one after another. Each frame of Star Wars ASCII art was hand made, and is 67 characters wide and 17 high. The animation is an ongoing (but sporadic) project started in 1997. The animation is currently the first 18 minutes of the movie and contains 15,953 frames which are displayed at 15 frames per second. blinkenlights.nl was kind enough to make Simon’s animation available over telnet.

by JP Brown

The solver first identifies the initial state of the cube using the camera and color recognition software written in VB5 (source available on JP Brown’s page). It then calculates a sequence of moves that will solve the cube. The move sequence algorithm was written by Mike Reid and later ported to Visual C++ for use with this project. It is guaranteed to solve the cube in 40 moves or less, but the normal case is around 20 moves. The Lego solver then uses the green or yellow grippers to rotate cube faces, or together with the blue gripper disengaged they can roll the entire cube 90 degrees. The blue gripper keeps the cube stabilized while the other grippers are rotating faces, and it can be used to spin the entire cube by 90 degrees.

The Naked Game

by RetroDev Games

The Naked Game is a Flash animation that pits two primitive artificial intelligences against each other in a game of Pong. The catch is that the Pong code controlling the mechanics of the game is visible below the board, and lines of code can be disabled and re-enabled to see the effect on the game in real time. The Naked Game is billed as the first piece of online conceptual video game art - built to explore the themes of freedom, restriction, and frailty. I’m not sure about all that, but it is a damn sweet hack.

Before diving into steps for improving job satisfaction, its worth mentioning the most obvious way to improve job satisfaction - switch jobs. Switching jobs is definitely something to consider if you aren’t very happy with your current position, and according to TechRepublic’s article on 10 signs you aren’t cut out to be a programmer, if you prefer regular raises to job hopping, you aren’t even cut out to be a programmer. So it seems that job hopping is a part of the culture and in many cases the only way to move forward in a programming career. A job change may be the optimal path, but it usually takes time.

Whether or not you are actively looking for a new job, it makes sense to always be looking for opportunities to become more satisfied with your current position. Right. Now. Why? Work is a big part of life. If you show up every day, you might as well do whatever you can to enjoy it. If you aren’t happy it will show in your work. This means lack of motivation, poor job performance, and bad references in the future.

Quality of Projects

1. Spend time working on projects you enjoy in your free time. Learn a new language. Contribute to an open source project. Read a book. Attend a conference. Start a blog. This has the added benefit of keeping your skills up to date and making you more employable.
2. Create side projects that are worthwhile and fun. Write some utility scripts to automate the deployment process. Improve the source control system with email notification of check-ins.
3. Ask to be transferred to another project. Given a choice sane management would rather transfer a star hacker than loose them altogether.

Work-Life Balance

1. Many hackers do enjoy the long hours and ability to focus all their energy on one task, but if this is your style it is important to take breaks after several days of long hours. Take a day off. Don’t hoard your vacation time.
2. Relax a bit and simplify your life.
3. Schedule time for family, friends, personal projects, and other commitments. Stick to the schedule.
4. Reduce your hours. Don’t be afraid to let management know that there are boundaries. They will respect you more for it. Really.

Bureaucracy & Politics

1. Rejoice in the fact that it could be worse.
2. Avoid gossip and rumors. Stick to your work and keep your head down.
3. Gracefully back away from situations that will result in more bureaucracy and politics.
4. If your manager is able to shield you from this, thank him. Profusely. Let him know that it’s appreciated.

Recognition & Respect

1. Become a mentor. You can do this on a more informal basis by leaving your door open to junior developers, or in a more formal way by joining a mentoring program.
2. Become a project lead. Don’t wait around for your company to promote you - start an open source project, or help out on an existing project.
3. Pick a niche and become an authority on it. Give talks at conferences. Blog about it.

Compensation

1. Ask for a raise. A nice way of doing this is asking for a performance review. Be prepared to highlight your achievements and specific ways you have contributed to the company’s success. Do some research on current salaries in your area and inflation rates and be prepared to quote them.
2. Barter for better health benefits, longer vacations, more telecommute time, free M&Ms with the green ones picked out, or whatever floats your boat.
3. Get a certification, go back to school, or take a few classes online.  Employers often increase compensation as education level goes up.

Teamwork

1. Get involved in the hiring process so that you can have a hand in building a powerful team. Hire those 5 star people and do anything you can to keep them happy.
2. Be glad you don’t work with Paula.
3. Extend your network outside of your current job to connect with like-minded developers. Join a LUG, programming association, or alumni group. Use LinkedIn.

Everyone goes though periods of dissatisfaction with their job. It’s normal. I’m no expert in employee motivation, but I have found that the above steps work for me. Hopefully you will be able to get some use out of them as well.

Photo by Massics

First lets break down the main indicators of job satisfaction, and look at how to measure satisfaction in each of those areas. In the next post in this mini-series I will write about ways to become more satisfied as a programmer.

Teamwork

According to Jeff Atwood, “The people you choose to work with are the most accurate predictor of job satisfaction I’ve ever found.” This rings true with me. Thinking back, during the times I was most motivated and happy with what I was doing, I was part of an excellent team of hackers. We worked well together. We bounced ideas off of each other. We were aware of each others strengths and weaknesses, and knew how to maximize the strengths while improving the weaknesses. We respected each others experience, knowledge, and all around hacker stardom. Well you get the idea…

Think of the best hackers you have ever worked with. Would they want to be on your team? If not, you are in trouble. If those hackers wouldn’t want to be on the team you are on now, its unlikely that your team will be able to attract other top notch hackers and its unlikely that you yourself are happy.

Good teams are made up of good hackers who work well together. Attracting top notch people is impossible without an environment that cultivates job satisfaction. So if the working environment doesn’t rank well for the satisfaction indicators below, it won’t attract good hackers, and therefore has virtually no chance of cultivating a good team. This is why the strength of the development team is the number one predictor of job satisfaction.

Quality of Projects

Intelligent people get bored doing the same thing all the time. Hackers are no different. If they are stuck with the same language, the same boring CRUD user interface, the same algorithms they learned in their first year programming, they will be unhappy. Most developers crave learning new things and being able to apply them. Difficult and challenging problems excite them.

Taking some time to think about the skills you have learned lately and the problems you have solved will give you a good idea of not only how satisfied you are in this area, but also how well you are advancing in your career.

Work-Life Balance

The hacker stereotype is to work incredibly long hours while surviving on Cheetos and Bawls soda. This death march method of software development isn’t sustainable and will take its toll over time, ultimately leading to burnout and job dissatisfaction.

Taking a look at the number of hours you are working will give some indication of how you rank in the work-life balance area. Many people on the track to burnout aren’t even aware of it, so talk to family and friends and ask them to help you gauge how well you are balancing work with other activities and obligations.

Bureaucracy & Politics

Some amount of bureaucracy and politics is unavoidable, but good management will be able to shield you from it to a great degree. The only time I found myself without it was during University, and that was more because of warped perspective than anything else. University bureaucracy was actually pretty bad, but I never thought of it as part of what I was doing. It just become something totally unrelated to programming; I thought of it more like paying the water bill or taking out the trash.

In general, happy, satisfied hackers are busy coding and are making progress towards a goal. As a group, programmers do not like to find themselves in endless meetings that accomplish nothing, ensnared in budget disputes, or without adequate resources to perform their best work. If projects are constantly stalled or held back by requirements that constantly change or are missing, management decisions, or lack of direction, hackers will be frustrated and unhappy. Worse, if programmers are left with nothing to do due to excessive bureaucracy and policy decisions, they will see that their skills are not being used and are therefore not valued. This leads not only to dissatisfaction with the bureaucracy, but also causes dissatisfaction with the amount of recognition and respect that they have.

Recognition and Respect

People who excel at what they do and are knowledgeable about a subject expect to be taken seriously and consulted with during decision making. This is as it should be. Management that disregards the opinions of their technical people or don’t consult them when it makes sense to do so will be left with uncooperative and dissatisfied developers when it comes time to implement an idea.

Have you been allowed to undertake difficult projects? Is your opinion sought out? Are your ideas taken into consideration? Are you congratulated for meeting important milestones? This are all good measures of your recognition and respect within a company. Are you often forced to implement something you don’t have any control over or disagree with? Are you often contradicted or marginalized by people who are less knowledgeable? (Careful with that one - make sure you are open to sound technical ideas.) These are indicators of dissatisfaction.

Compensation

This is one of the easiest factors of job satisfaction to quantify. Some quick research at online job boards gives a good idea of compensation packages for similar jobs in the same area. Discovering compensation packages for people in you own company can also be valuable information. Obviously if you are under-compensated you will not feel satisfied, and on the flip side if you are compensated well you rank as highly satisfied on this area.

After looking at and evaluating these criteria one by one, I have a much better understanding of my personal job satisfaction. It is easier to see which areas are working well and which I need to take action on in order to improve my own career satisfaction and general well-being. In two weeks I will cover steps to increase job satisfaction for each of the factors listed above.

In the meantime, what factors are important to you?

Read the next post in the series: Become a More Satisfied Programmer. Today.

Background

E2 has 256 square pieces that must be placed on a 16×16 grid. Each puzzle piece has 4 edges, each with one of 17 possible patterns, which must match the pattern of the neighbor piece that touches that edge. Each of the edges touching the border must be gray. There is also one hint piece which must be placed on the board at the indicated position and orientation.

The animation below shows a methodical approach to solving a 4×4 puzzle with no hint piece. Pieces are taken from the right side and placed on the board on the left. When a dead end is found and no more pieces can be placed on the board, pieces are removed up until the last decision point and a new piece is tried at that spot. The algorithm you are seeing in action is a backtracker.

Clues & Rules

An additional hint is provided for solving each of the 4 clue puzzles, for a total of 5 possible hints. E2 clue puzzles I and III are 6×6, and E2 clue puzzles II and IV are 6×12. Clue puzzles III and IV are only available in the UK as of this writing. It is not required that the E2 solution use these optional hints, and it is suspected that E2 has more than one possible solution.

In order to win the $2 million prize, submit the solution to the Eternity 2 adjudicators by December 31, 2008. On the 31st, all entries will be opened and the entry that was submitted first wins the $2 million. If nobody submitted a complete solution, the competition will continue until December 31, 2010. A smaller prize may be awarded for the highest scoring partial solution if no complete solution was submitted. The complete rules can be found on the official site.

Sounds easy right? Well…

Solver Implementations

A few solvers have been made available online in the files section of the E2 Discussion Group. I picked two of the most straightforward solvers and ran benchmarks on them.

The first solver I looked at is Doc Smith’s C++ solver which is based on a solver by Marc Lebel. Doc’s solver works by first creating arrays that list the pieces that will have given colors on certain edges. This means that finding a piece with, say a green left edge and a red top edge, requires indexing the 2 dimensional array storing left and top matching pieces with the code for a green edge and the code for a red edge. The solver works by starting at the top left corner and tiling towards the bottom right corner (just like in the animation above), so there is no need for an array that would contain matches for say, only top and bottom edges. If the solver reaches a dead end, it backtracks recursively to the last decision point and tries again. Once the solver reaches the lower right corner, a solution has been found.

Next I looked Joel’s solver which is a Java port of the C++ backtrack solver written by Doc Smith. These two solvers work the same way; they differ only in the implementation language.

The E2 Discussion Group is very active and contains a lot of information on different solving strategies, so if you are interested in more advanced algorithms, have a look there.

Benchmark Results

I ran benchmarks on both solvers to compare their performance. The benchmarks are puzzles similar in design to E2, but of a smaller size. I ran benchmarks on 6×6 with 1 hint, 8×8 with 2 hints, and 10×10 with 3 hints puzzles to get an idea of what kind of complexity we can expect from E2. All of the benchmarks were run on a dual core 2.00Ghz Intel with 1 GB of RAM. No compiler optimization flags were used for any of the test runs.

Here is a table giving the mean solution time for each of the solvers (the benchmarks had more than one possible solution, so I used the mean time in the table). On the 10×10 benchmark, I stopped the test after 10 hours. Neither of the solvers had found a solution at the time. The complexity of the puzzle increases exponentially in the number of pieces - this is to be expected of an NP-Complete problem.

One mildly surprising result is that the Java solver outperforms the C++ solver. Enough to make a difference solving the 16×16 E2 puzzle? Not likely. At this rate it would take more than a lifetime to find a solution for the 16×16. Improving the solving algorithm will give much better gains than code-level optimizations.

So will you get rich solving E2? Well, somebody will. If this type of challenge interests you, its definitely worth a shot.

Resources:

Purchase E2 from Amazon

Joel’s Java solver

Doc Smith’s C++ solver

E2 discussion group

[Disclaimer: Links to amazon.com are affiliate links.]

The Mythical Man Month: Essays on Software Engineering - Anniversary Edition

by Fredrick P. Brooks

This is a classic on the human elements of software engineering first published in 1975. The technology has changed a lot in this time, but the human elements of software engineering have remained the same. It is a wealth of insight, often quoted, and very well known in the industry. “The Mythical Man Month” describes many commonly occurring problems in large and mid-scale development projects and breaks them down. Here are a two of the book’s important principles:

The Mythical Man-Month: Adding manpower to a late software project makes it later.

No silver bullet: There is not a single strategy, technique, or trick that can exponentially raise the productivity of programmers.

I recommend this book not only for programmers, but for anyone managing a software project. Project managers and programmers alike will appreciate Brooks’ clear, well-thought out points.

The C Programming Language (2nd Edition)

by Brian W. Kernighan and Dennis M. Ritchie

Commonly referred to as just K&R, this is the canonical C reference book. It’s to the point without being too terse; its detailed enough for a beginner to understand without being bloated. K&R tells you exactly what you need. Nothing more, nothing less. At 274 pages this is one of the most compact languages references you will find. I dare a Java author to come up with something so sweetly concise.

This book is recommended for anyone learning C, and for anyone looking for a C reference book. K&R is a must read for anyone who is writing a language reference or technical literature.

If you have never programmed before, K&R might not be the best place to start out, but still doable if you are motivated. Beginners might also consider getting a copy of “The C Answer Book,” which gives detailed explanations of solutions to the K&R exercises.

Structure and Interpretation of Computer Programs (2nd Edition)

by Harold Abelson and Gerald Jay Sussman

SICP is used in entry level computer science classes at many top tier universities including the University of California - Berkeley and the Massachusetts Institute of Technology. It uses the Scheme language to introduce many powerful paradigms like recursion, lambda notation, abstraction, and interpreted languages.

I found this book incredibly dense when I first read it for an introductory CS class, and there was still much to learn from a reread several years later after I had a better grasp of the ideas presented here.

If you didn’t have the opportunity to use this book in the classroom, I would recommend picking it up to see what you missed, especially if you haven’t used any of the Lisp dialects extensively.

Code Complete 2: A Practical Handbook of Software Construction

by Steve McConnell

“Code Complete 2″ is a highly regarded book about software construction, where McConnell defines construction as mainly programming and debugging, along with some elements of construction planning, detailed design, unit testing, integration, and integration testing.

The book’s focus is on writing better code. McConnell touches on a wide variety of topics including managing complexity, refactoring, coding style, and writing good comments.

This book is recommended for anyone who wants to write good solid code. It will save beginners time when learning good coding practices, and is a great refresher for programmers with years of experience who may find that over time they have developed habits that are holding them back.

Introduction to Algorithms

by Thomas H. Cormen, Charles E. Leiserson, Ronald L. Rivest, and Clifford Stein

This is another textbook. “Introduction to Algorithms,” is probably the most popular university-level text for algorithms classes. It also serves as an excellent reference book.

Even though practically speaking most programmers shouldn’t be writing their own implementation of, say, quicksort, in a production environment, algorithms knowledge is essential for understanding what kind of asymptotic performance you can expect from libraries and your own code.

“Introduction to Algorithms,” is recommended as a reference for any developer looking to brush up on their algorithm skills. It is also recommended if you are interested in topics like NP-Completeness , randomized algorithms, or Fast Fourier Transformation. Good math skills are essential if you want to grok all parts of this book.

Design Patterns: Elements of Reusable Object-Oriented Software

by Erich Gamma, Richard Helm, Ralph Johnson, and John M. Vlissides

This is a catalog of different object-oriented design patterns. For each of the 23 patterns that are presented, the authors give an in-depth analysis including a motivating example for the pattern, common pitfalls and trade-offs, related patterns, and sample code in either C++ or Smalltalk. Some of the patterns covered are Singleton, Observer, Template Method, Iterator, and Proxy.

This book is a must read for intermediate or advanced developers working on object-oriented code. Coders that are new to object-oriented design would be best served by mastering the basics of an object-oriented language and digging in to non-trivial project before attempting to learn all of the patterns presented here, otherwise the patterns aren’t likely to “stick.”

Programming Pearls (2nd Edition)

by Jon Bentley

This is a great book for learning how to approach problems. Each of the 15 chapters presents a different task, such as sorting phone numbers, creating anagrams, or doing text searches. The problem is defined, and then Bentley walks through different solutions, providing a discussion and clear analysis of each solution, with a breakdown of programming principles at the end.

“Programming Pearls,” contains some very clever ideas, and would be of some use as an algorithm reference for a small class of problems, but the take away of the book is something much bigger and more general: how to approach and solve problems from the perspective of an engineer.

I recommend this book to anyone who enjoys working though programming brain teasers.

Compilers: Principles, Techniques, and Tools

by Alfred V. Aho, Ravi Sethi, and Jeffrey D. Ullman

“Compilers: Principles, Techniques, and Tools,” is commonly known simply as the “dragon book.” This is the standard textbook in a theoretical compilers class. It covers everything you would expect to find in a compilers text: language syntax, parsers, lexical analysis, grammars, intermediate code generation, runtime environments, optimization, etc. If you haven’t done a lot of work with these topics, be prepared to learn a whole slew of new termonolgy and concepts: deterministic finite automata, finite state automata, LR parsing, and more.

I particularily enjoyed parts of the dragon book that get down and dirty into the mathematical notation and theory - and this is large theme throughout the book. If you find regular expressions and state machines facinating you will really enjoy the book. On the other hand, if you are looking for a text that will introduce modern, practical compiler implementation details you may find the dragon book a bit lacking. In those respects the dragon book is starting to show its age (this version was copyrighted in 1986).

Unix Power Tools, Third Edition

by Shelley Powers, Jerry Peek, Tim O’Reilly, and Mike Loukides

This is one of the many great books from the O’Reilly collection. “Unix Power Tools,” weighs in at a hefty 1200 pages, and is full of descriptions of Unix command line tools, usage cases, and shell script syntax examples for all of the major shells.

“Unix Power Tools,” is a great reference book and is fairly well organized into self-contained chapters. If you find yourself reading this book from cover to cover you are an amazing specimen of geekdom.

Much of the information contained within this book can be found more quickly with a Google search, but sometimes its nice to have a dead tree reference as well.

This is one of the few books that is valuable for beginners and experts alike. Everyone from a complete Unix newbie to a seasoned Systems Administrator has something to gain from “Unix Power Tools.”

The Ultimate Hitchhiker’s Guide: Five Complete Novels and One Story

by Douglas Adams

Well this book isn’t technically about hacking, but your geek card is hereby revoked if you haven’t at least read this “increasingly inaccurately named,” Hitchhiker’s Trilogy. It contains all 5 books in the Trilogy: “The Hitchhiker’s Guide to the Galaxy,” “The Restaurant at the End of the Universe,” “Life, The Universe and Everything,” “So Long, and Thanks for All the Fish,” and “Mostly Harmless,” plus the short “Young Zaphod Plays it Safe.”

Be entertained. Discover the answer to life, the universe, and everything. Learn how to make a Pan Galactic Gargle Blaster. And most of all don’t panic!

This edition is bound in beautiful black leather and has a silk ribbon bookmark sewn into it. It’s like reading a very nice Bible, only more believable. (Mandatory disclaimer before any trolls find me: That was a joke.)

That’s it for my top 9½. What would you put in yours?

[Update: Have a look at the follow-up list, 5½ more books in a hacker’s bookshelf.]

[Disclaimer: The links above are affiliate links, which means if you buy something from Amazon after following a link from GrokCode, you pay the same amount and I will get a percentage of the sale to help support this site (which would make me smile).]

I am making Sweet Hacks a regular series here at GrokCode, turning it into a blog carnival that runs every three months. If you would like to nominate your own sweet hack for the next issue, send an email to jess [AT] grok-code.com with a short description of the hack and a link. Or if you are a del.icio.us user, tag the page “for:grokcode” to put it into my “links for you section.”

1. Shamus Young shows us how to create random cartoon-like faces in PHP. It works by using the GD library to fill an image with a random hue for the background, then randomly chooses and inserts a face shape, eyes, mouth, and other facial features into the image. Here is the full description of the image creation process. The resulting images are used to create avatars used in the Wordpress Wavatars plugin.
2. Next up is codepad by Steven Hazel which allows you to paste code in 10 different languages including C, Perl, Ruby, and Scheme. Codepad then compiles and executes your code under ptrace in a chroot jail inside a virtual machine. Steven also describes how he was able to scale codepad after a 1000x increase in traffic on his blog hackerdashery.
3. Simon Willison shows us how to combine semantically marked html with the jQuery JavaScript library and Google Maps to create one of those new-fangled web 2.0 mashups. Have a look at his great code walkthough to view the results - a map of restaurants in Brighton.
4. Ever had to hack something together that ran faster than grep? James Hicks has. And he lived to tell about it on his blog isnerd.net. James uses some interesting properties of the data he is matching to write a lighting fast matching program in C.
5. Last up is a shameless plug for my Joke Generator written in Common Lisp. You seed it with a vocabulary and it tries to use this knowledge to create jokes of the form “What do you get when you cross X with Y.” Hilarity ensues! Although sometimes its more the “What?!?” kind of hilarity than the ha ha kind.

Well thats it for this installment of Sweet Hacks. Don’t forget to let me know about your own hacks.