How to do a large piece jigsaw puzzle

Almost everyone has done a jigsaw puzzle but the strategies for solving small puzzles don't work very well on a large puzzle. The largest puzzle I've ever done is 5000 pieces. Frankly that's too big. You need a large work space. I had to do it on the floor. 

My favorite size is 1500-2000. You can solve these puzzles by hunting and pecking but most people find that gets tedious fast and can take a lot of space. You need to have a plan.

I like to divide and conquer, making this a suitable post for my tech blog.

The puzzle I'm using for this post is one I picked up recently at a thrift shop. I've never done it before. It might have missing/damaged pieces. This post is a stepwise breakdown of the build. None of the images were set up. This is exactly how I do a build.

This puzzle has four main areas: sky, water, boats/houses and trees. It is a difficult puzzle because both the water and sky areas are sizable and the boats and trees lack large features - a piece of boat, house or trees could go in several different areas.

This puzzle needs a divide and conquer strategy. Imagine a scenario where someone gave you just the pieces of the stern of the "Adler" boat prominent in the foreground. That's a much simpler problem. Most people would have no problem putting that together. That's the value of divide and conquer, breaking a big problem up into smaller problems.

The first and ongoing task in my approach is separating the pieces. I pluck pieces one by one from the box and sort them into piles. Some piles that I know I'm going to only tackle later don't need to be neat, face up - they could go in a bowl, e.g. the water.

I start by scanning the box. Usually something stands out visually. I sort these first - they are the easiest to see. The sky includes a lot of pieces that contain bits of mast, trees, etc. I want all pieces that contain any sky. The interfaces between the sky and other objects can be one of the easiest parts of the puzzle to tackle. For now I sort them all into one pile.

Part of the water contains reflections. For now all I want are the featureless water pieces. Some of the reflections might be hard to tell apart from other dark areas of the puzzle.

While sorting, other things begin to stand out: the red ball, the green house, wording on the boats, the yellow mast with the green rope, the blue rope. These features have relatively small numbers of pieces and I assembled some of the them while I was sorting.

As sorting continues, I start to notice the trees and white pieces. These go in more piles. Edge pieces go in a pile. Here are pictures of my first sort. When I stop easily finding pieces for any of these piles, I stop.

There's a lot left in the box at this point but I can start putting the puzzle together. Obsessing on getting all of any pile is inefficient. Alternating between sorting and assembling will tune your eye.

For example, with the first sort, I was able to assemble most of the red ball, the red car, the green house and some of the boat hulls at the water line. 

Then I scanned the box and found a few more pieces with red. Every time I look in the box for a certain kind of piece I continue to look for pieces that go in all of the other piles. So, over time, each pile becomes more complete.

Now that my eye is tuned into the red, particularly, the water with red reflections, I am more likely to notice them in the box. I don't need to focus on finding them all now. My eye won't be able to help noticing them while I'm scanning the box later.

While sorting I noticed the blue line atop the stern of the foreground boat and the lettering below it and started pulling those. Sorting is a continuous process.

That's pretty much it for the low hanging fruit. Next I needed to decide which pile to start with.

Commonly people start with the edge pieces first. I usually don't. In a big puzzle, it doesn't help much. This puzzle is 52 x 39 (2028 pieces). The edge is 178 pieces, 8.8% of the puzzle. 

Compare that to a 16 x 13, 208 piece puzzle where the edge is 54 pieces, 26% of the puzzle. You do the edge and your well on the way to done. But in the big puzzle:

• the edge is harder. Some pieces may fit in more than one place. With half the edge being sky and water it may be hard to notice errors.
• some edge pieces may still be in the box. Remember, we aren't obsessing about finding every kind of any piece. A single missing piece breaks the edge making assembly difficult.
• assembling the edge takes up a lot of space and cramps the work space.
• assembling the edge doesn't help with the rest of the puzzle. It will later but early on it doesn't.

I started with the white pieces pile and the small easy piles (green, red ropes, etc.). Some of the edge pieces obviously fit with these so I pulled them from the edge pile.

I considered looking for more white pieces but those pieces are hard to place. They are interspersed with a lot of other pieces. I thought about the trees but they would be a lot easier after the houses/sky are done and create a context for them. So I started with they sky - the sky pieces that interface with the tree line, masts, etc. I first separated the plain old sky pieces from the ones I was interested in.

The plain old sky is a challenge but the masts, etc. were surprisingly easy. because the sky is so easy to spot in the box, I was pretty sure at this point that I had most of these pieces. Above is my first pass (yes, its upside down - that was my working perspective).

Notice that I pulled the relevant pieces from the edge pile. I was happy with how easy it was to assemble masts so I started pulling the large yellow mast pieces (left side of picture). Finished it looked like this:

Next i did the yellow masts, more of the white pieces and roof pieces.

Then a reorg of these three main puzzle areas to put them together. Since I'm working in a space barely bigger than the puzzle, I need to shuffle the work space around.

Now its pretty easy to place the rest of my white, roof and tree piles.

Now you can see the bottom of the box.Mostly water and dark pieces which I sort into three piles - dark pieces, water with reflections, remaining house, boat pieces.

These pieces are the dregs. They aren't easily identifiable as anything specific - they didn't fit in any of my previous piles. However, the houses, boats, trees, etc. went together pretty easily because the puzzle was mostly complete - I only had to scan the empty holes. There was one missing piece (in the trees, by the mast at the treeline). With a used puzzle you're never sure how many pieces will be missing. Another good reason not to obsess about finding every piece.

All that's left are the real dregs - the featureless dark pieces, sky and water. I retrieve the featureless sky and water pieces that I had previously set aside.

Finishing these featureless areas of a puzzle is an art of its own. It is too tedious for many people.

These puzzle pieces have subtle patterns in shape and I was able to separate them into horizontal and vertical pieces. Not 100% accurate but it divides the pool roughly in 2 piles. These pieces have some pretty unique shapes so many of these pieces could be matched by shape.

At this point some trial and error is unavoidable. These pieces have no useful features - no texture, color variation. Nothing. You need every trick you can find to reduce the number of pieces you need to try.

The left edge of the puzzle had a consistent cutting error so those pieces were easy to spot. Like I said, you need every trick you can find...

Here's the finished puzzle. In addition to the missing piece, there are two damaged pieces bottom left.

It was quite a difficult puzzle. I didn't keep a time record but it was built in roughly 10 sessions of 1 to 2 hours. Each step was a sort or assembling a sorted pile (or piles). It was built in a space barely bigger than the puzzle.

I believe the key is solidly in the sorting process. If i were to give someone the pile of pieces for any puzzle object they can probably assemble it.

If you can accurately distinguish pieces by color, texture, pattern you can divide and conquer. I'm able to sort quickly and accurately so I'm able to enjoy a puzzle that would otherwise boggle.

I've found that, like many problems in life, you can't tell exactly how you are going to solve a puzzle. It's a process, not an algorithm although individual steps may be algorithm-like. Sorting achieve two key results. It gives you a pile of pieces that comprise a thing (or at least similar/indistinguishable things). Some of these piles you want because they have structure that you can easily assemble. These you are going to build early. Others have no structure. These you are going to build last. After the shape of the rest of the puzzle is done and you have a clear area with all of the boundaries known - so you can rely on trial and error if you need to.

Perhaps the most important benefit of sorting is that every time you remove pieces of a clearly visible type from the box you make it possible to see other categories of pieces. In engineering terms you can think of it as removing noise. When you take all of the bright blue sky pieces out you can start to see more subtle patterns.

Sorting can also be recursive. Once you sort a pile you may notice that it contains a clear subdivision of pieces. Like I did with the sky - plain old sky and sky with masts, etc..

The eye is very good at seeing patterns  I'm lazy so my approach lets my eyes do most of the work.

A final point I will emphasize is that this approach also doesn't need to be 100% accurate. A good sort is more efficient but a sloppy sort will still work. With practice the sorts will get better. It is actually critical to not agonize over the sorts or it will be no fun at all. And it's best not to obsess. With this strategy you don't need all of the pieces to start assembly. Stray pieces will stand out like a sore thumb. For instance, after you've removed the sky, a bright blue sky piece will jump out at you - and you'll know exactly which pile it goes in. Or, better yet, if it's a distinctive piece, where it goes in the puzzle.

C# 6.0: Many nice features but dependency chain is very limiting

Microsoft has done it again!

I've been dabbling with C# 6.0 features and I am really liking it. However, Microsoft has made adoption a real problem.

C# 6.0 is only supported in Visual Studio 2015. Ok so far but Visual Studio 2015 will only install on Windows 8 or later.

Gack!

Windows 8 was stillborn and no one wants to install it. Microsoft tried to convert its Windows desktop OS to a tablet OS and made a spectacular mess of it. So much so that they hastily released a Windows 9 (which they called 8.1 so they could offer it as a free upgrade). So dramatically didn't it fix things that they have now also offered Windows 10 as a free upgrade. People still don't want it so they have now taken the rather extreme measure of dramatically shortening the support agreement for Windows 7.

However, the whole mess of convolving the tablet OS with the desktop OS has left everyone afraid. Microsoft is trying to make the upgrade even easier (update is becoming more like a service pack) but many people that have Windows 7 aren't taking the bait.

In my office we are mostly still on Windows 7 and people are not eager to install new Windows versions. A few nice syntactic features in C# 6.0 aren't going to push people over the hump.

So, I've had to abandon C# 6.0 for now. I imagine this experience will be common and that adoption of C# 6.0 will be weaker than it should be.

To fix this Microsoft could decouple VS 2015 from Windows 8 or back port C# 6.0 into VS 2012. Otherwise, they risk losing momentum with what is really a great language and development environment. Come on guys, give us some better options.

Use Commit Discipline to Limit Code Entropy

Source control software is a basic tool that we are all familiar with but it is often perceived most importantly as a management tool to associate work with bug/feature tickets and manage deployment to release branches.

It is also a very useful for keeping code clean which can be especially important if you have mission critical code areas.

It is very simple and not particularly time consuming: look at all the differences when committing code and make sure your commit is clean. If your code base is clean you want to keep it that way and to aid troubleshooting of new bugs you want to be able to trace the change history.

One common scenario is that a change has required experimentation and debug logic. Once you finalize an approach you may have remnants of experiments, debugging logic and fragmented logic. Review this code from the perspective of the finalized approach. Ensure it is clean. It is surprising how often I've avoided committing vestiges of the development process.

Keep the granularity of commits as small as possible. It is a great way to focus on the change in context. For example, as part of a new application feature you've added a new method to your Rectangle class. Reviewing and committing the Rectangle change separately creates a focus on the Rectangle class that will help to maintain the intent and overall design of the Rectangle class.

Avoid coupling refactoring and gratuitous changes that obscure differences with functional changes. Implement refactoring and cosmetic changes as a separate commit and note in the commit comment that there are no functional changes. A simple example is moving code around. As a class grows it can help to reorganize the code layout but most source control software sees these moves as differences. If there is an actual difference in the function you have moved it is likely you won't notice it.

It's also a good time to review the area around your change. Are there code comments that are affected by your change? Have you obviated or orphaned existing code or variables?

Keeping mission critical commits clean is a very simple way to keep your mission critical code and its history clean. The second law of thermodynamics applies to all systems, including a software system. All changes will tend to increase entropy (disorder) but we can minimize the disorder by the these simple practices.

You don't have to be as disciplined everywhere but I find that developing this simple discipline where it is most important becomes habit forming and helps improve the quality of all commits.

Being on the Wrong Side of an Arrogant Tech Idealist OR How To Be A Hired Gun.

I had a miserable experience with a new hire, a self professed object oriented guru. I wasn't involved in bringing this fellow in. An odd situation but I approached it with an open mind. A fresh pair of eyes can be a good thing. It didn't need to be a miserable experience.

It made me think about how one should go about evangelizing change. I've been on the other side of this fence but I've never seen how much the approach can dictate the outcome.

The first problem was that management gave this person carte blanche. Not a good idea if you want to avoid conflict. However, the guy should have known better than to abuse this privilege so that doesn't change the lesson of this post.

First rule. Be sensitive and diplomatic. Don't be shocked or indignant by anything you see. Small companies have many skeletons in the closet. That isn't necessarily a reflection on the people, the very people that did the very work you are critiquing. If you put people off they won''t want to help you.

Listen. People know a lot more about the technical and cultural challenges than you do. Especially listen for consensus and push back. Focus initially on issues that have consensus. Build rapport by building on consensus. Defer sensitive issues until you have built trust and can explore the reasons for the push back.

Don't talk down. Some of the things you advise may already be well understood. Constraints may have dictated the course of events, not knowledge. Also, people may want to learn new practices but that will dry up fast if you talk down to them.

Be pragmatic. This blog is about software development in small companies. There is not much room for idealism in a small company. Demonstrating pragmatism will earn you respect. Revealing idealism or dogmatism  is a sure way to turn people off.

Contribute. Do a feature. Fix a bug. There's no better way to learn about the organization and earn other programmer's respect.

Compartmentalize. Practical examples will be necessary but be selective when demonstrating or applying your recommendations. Don't start out by refactoring everything you touch. Instead, focus your demonstrations on your own contributions or pick a specific area in the system. Look for some level of acceptance before expanding your scope.

Be frank. Don't be afraid to admit that there will be some pain. No pain, no gain. Change always has a cost.

Be firm. Don't sit on the fence. If you decide to go a certain direction, go.

Pick good demonstrations. Make sure there is a tangible benefit. Remember, you are dealing with pragmatic people. If practical demonstrations are not easy to understand and compelling you may not get buy in.

Respect current practices. For example, if there are code reviews, offer your code for review. Don't set yourself above the status quo.

Pace yourself. Oversaturating people will turn them off. Think evolutionary rather than revolutionary - revolutions can be bloody and we're trying to avoid that. At the same time, avoid going too slow or people won't pay attention.

Know when to stop. Keep the costs and benefits in mind. Even if you've been successful so far, don't forget that you can still lose people if you forget to be pragmatic.

The approach might be different if you are a consultant.Then you might have a much shorter time frame. However, the playing field is the same. If you want your recommendations to be followed through you still need to demonstrate value and earn respect.

Find Out What People Find Rewarding

Job satisfaction is critical to employee retention and productivity but companies often misunderstand what people find satisfying. There isn't a simple answer because people are different so it is very important to understand your people.

One thing for sure is that while money can be a factor in retention, it is not a very good source of job satisfaction. In a frustrating environment, you might hear a common cliche, "Oh well. Another day, another dollar." If you hear that, you are in trouble. Money may be some consolation when you have had a crummy day but it is a meager consolation.

There is a dramatic variance in productivity. People can be many times more productive when they are motivated and satisfied than otherwise. It can really pay to understand what gives people satisfaction.

Junior and senior people have different needs. The needs may be similar but the scope will vary. Senior engineers need to see how their work contributes in the big picture and see that their recommendations and designs have visibility. Junior people need to see how their work fits into the part of the system they are working on and that their feature/piece is appreciated.

There are some common factors in job satisfaction and dissatisfaction:

• challenge, coolness
• visibility, appreciation.
• appropriate forms of support
• quality
• pain level

There are some common scenarios that can be satisfying or dissatisfying depending on how they are handled. Almost everything management does has some effect on individual people's job satisfaction and motivation. Be aware of the effect of your management decisions and style.

Challenge

Most software involves a certain degree of tedium. People who have significant responsibility for maintenance work, backwards compatibility, etc. may have a high degree of tedium. Don't forget to share out some of the cool work to all people. At least give them a chance to be involved when there is new/cool work. Don't get caught in the trap of thinking that a person is too busy and that you can't afford to have them work on something cool. 

Avoid a hero culture when a select few get all of the cool work and visible appreciation. Instead, create a culture where everyone contributes and gets visible appreciation.

Avoid a "give it to the Co-op" culture where work that would be interesting to a senior person is given to an overly junior person, especially when you will later expect the more senior people to live with and possibly maintain the thing. Instead, give the Co-op to the senior person to direct and mentor. 

One of the best experiences for junior people is to work with more senior people. You can easily spoil them by giving them a taste for glory too early.

Appreciate

Show appreciation at the level where the work is done. Regularly. As immediately as possible. If the only visible appreciation given is the open e-mail from the CEO congratulating engineering on a project that just achieved a General Availability release milestone, don't be surprised if this fails to make people feel appreciated. It's too little, too late. It could be months to many months after the work was done.

Appreciation should be based on tangible evidence that the work is complete. A unit test, integration test or demonstration. Don't be arbitrarily appreciative or you will encourage poor work.

Recognize initiative. If someone shows initiative in a valuable area, give them visible appreciation, give them a role moving forward. A financial bonus for initiative can be a very much appreciated and a better use of money than a raise.

Don't bypass people. If they've had a role in the past, involve them in the future.

Support

Listen to what your best people need. Be careful not to intrusively, forcefully help them, like taking a part they've done good work on and give it to someone else because you think they are too busy. Talk to them first. Find out what they want. 

If people ask for help, listen.

Quality

Have a quality agenda if possible. Allow people to feel proud of the product and their contribution to it.

Pain Points

Develop a good development culture. Recognize competency and allow your best people to work with your best people.

Some development activities are painful. Don't try to hide necessary evils but do try to minimize any unnecessary pain points. Avoid ADHD behavior, militant project management, finger pointing, etc.

Emergent Challenges

I recently read "Prey" by Michael Crichton which is an exciting, easy read that explores the surface of nano-tech programming gone wrong.

A key element of the story is that high level goals and behaviors are too sophisticated to program into such simple devices. A programmer devises individual behaviors that interact to result in a desired collective, overall, emergent behavior. This is also how insects work. No individual bee knows how to build a hive. It is the outcome of a collection of primitive individual behaviors.

Where it goes wrong is in how difficult it is to predict and control emergent behaviors.

This idea of emergent behavior exists in more complex systems. In fact, I imagine it exists in every system. This article is about how it exists in technology projects in a real way that can affect quality, time to market and success or failure.

Pointers are a powerful feature in C but so many problems emerge from programming using pointers that modern languages like Java and C# have completely abstracted pointers and object life cycles out of the language and moved them into a virtual machine with automatic garbage collection. A solution was too dependent on garbage collection to be solved with pointers. Solving the emergent pointer problems of C required a paradigm shift.

Sometimes emergent problems could be solved in the original domain but laziness and new features often drive new paradigms, e.g. Microsoft's shift from WinForms to WPF. Although it is a paradigm shift in some ways I don't think the advantages would be compelling to most small companies. What Microsoft doesn't tell you is that, while WPF may be the answer to some of the problems that emerged from WinForms, a whole new set of emerging problems will be caused by WPF. Here is one of the most sensible things I've seen written about WPF - http://stackoverflow.com/a/75387.

Every new technology will have new emergent challenges. Don't forget this simple fact when tempted to switch to a new technology. The real problem is that a new technology is new to everyone and the problems that are going to emerge aren't yet known. Also, the evangelists are biased. Don't expect them to caution you about realities. Think about how aggressively Microsoft evangelized COM. Where is it now?

Even large scale systems have emergent behaviors. For instance, the obsession with copyright in the western world has driven a lot of innovation into the far east. We arm patent owners with trivial or self-evident patents and powerful legal leverage that prevents derivative innovation. The emergent result is to empower countries where those patents are not enforceable to innovate and threaten our dominance in a technology.

Another example is the success of personal injury lawsuits in American society. Bell used to be a prominent motorcycle and bicycle helmet company. Because they were American they were the successful target of anyone in the western world that wanted to sue them after personal injury because they were wearing a Bell helmet. The result was to stifle the helmet industry in North America and create a vacuum which was filled in the Pacific Rim where personal injury lawsuits could not easily get traction. For years the motorcycle industry has been dominated by Arai, Shoie, HJC, etc.

Be wary of new paradigms. Even if the end result is compelling, the transition will almost certainly cost more than anticipated, take longer the estimated and present a new set of problems that will take further time before new practices are established and incorporated in your development environment.

My best advice is to be aware of new techonologies and to experiment with them so that you can judge the cost/benefit issues for your own situation. When you want to transition, do it in a series of phases that mitigates the cost, learning curve and risk to delivery dates for your next few releases.

Lessons Learned and Not Learned

A start up takes many expediencies. As it starts to grow and especially when it brings in new people in management, product management and operations roles there will be need for change. I have seen many companies review operations and tabulate lessons learned. In theory this is a good exercise but there are several pitfalls.

Some Constructive Ideas for High Level Corporate/Departmental Reviews

Lessons learned are often symptoms. This is one of the biggest pitfalls. Patching symptoms can be very tedious and expensive because they tend to pop up everywhere. Make sure that your lessons learned are the right lessons. Try to understand the root causes of perceived problems. Addressing one root cause will likely address many symptomatic problems. Avoid the pitfall of essential lessons NOT learned.

Be respectful to the people that got you to where you are today. They are the doers. Be careful when introspecting not to demoralize and demotivate the people that actually know how to get things done. Don't talk about the current state and past actions negatively. If you believe that your problem is competency you need to fix that before doing any kind of reviews or post mortems.

Consider that the lessons learned might be wishful thinking. Don't encourage wishful thinking and whining. Don't legitimize it. A good sign that your lessons learned are whining is that they are symptomatic of some underlying issue, like limited resources.

Don't get stuck in this introspective, critical mode. I've seen companies perpetually talking about lessons learned. For years. If you embark on self analysis, give it a specific time frame and make specific conclusions and actions and then stop.

Don't make a laundry list. If you have more than 5 lessons learned there is something wrong. You might notice many issues but filter that down to no more than a handful of lessons.

Don't undermine your strengths. What some people consider a weakness might be the result of a very important strength. Fixing a minor weakness might have serious side effects. Be careful not to lose the spark that got you where you are today.

Be wary of solutions. Every solution makes the environment more complicated. Every complication adds new problems. Especially be wary of adding controls and processes. They are often more difficult to do well than they appear.

Be wary of silver bullets. Real world problems rarely have trivial solutions.

Be balanced. Post mortems are often top down - managers, department heads. Nearly all of these people have the same perspective. Often your engineers know the root causes and best practices to address the real problems that you face. If your lessons learned are all processes, controls, etc. then there is a good chance that your analysis is one sided.