Other writing in March 2017

Where have I been writing when I haven’t been writing here?

The Next Platform

I’m freelancing for The Next Platform as a contributing author. Here are the articles I wrote last month:

  • Solving HPC conflicts with containers – I finally understand the benefits that containers bring to HPC specifically.
  • Apache Kafka gives large-scale image processing a boost – Modern camera technology can produce images too large for a single machine to process in real time. This is where software solutions come in handy.
  • Peering through opaque HPC benchmarks – If Xzibit worked in the HPC field, he might be heard to say “I heard you like computers, so we modeled a computer with your computer so you can simulate your simulations.” But simulating the performance of HPC applications is more than just recursion for comedic effect, it provides a key mechanism for the study and prediction of application behavior under different scenarios.
  • Serving up serverless science – “Serverless” is a big deal in the general-purpose cloud, but can it work for scientific computing?
  • Increasing HPC utilization with meta-queues – HPC resources tend to favor very large jobs. One researcher found a way to get better treatment for small runs.


Over on Opensource.com, we had our 6th consecutive million-page-view month. This is getting to be old news. I wrote the articles below.

Also, the 2016 Open Source Yearbook is now available. You can get a free PDF download now or buy the print version at cost. Or you can do both!

Cycle Computing

Meanwhile, I wrote or edited a few things for work, too:

Bulk removing invalid emails from Salesforce

Hey, who ever thought they’d see a Salesforce how-to on this blog? One of the things I do at work is put together our newsletter and send it to our leads and contacts. But our list of contacts has built up over the years and some of the email addresses are no longer valid. So here (largely for my own reference later) are the steps I use to clear out the invalid emails. There may be an easier way, this is just what I’ve figured out.

  1. Export bounced emails from Constant Contact
  2. Put bounces in a spreadsheet tab called “CC”
  3. Export your Leads from Salesforce (just get the ID and email fields)
  4. Put the Leads into another tab on the spreadsheet
  5. Use this formula to identify addresses that need to be removed (because they exist on the CC tab):
    =IF(ISERROR(VLOOKUP(B2,CC!$E$2:CC!$E$2000, 1, FALSE)),"", "REMOVE")
  6. Sort by ColumnC Z->A
  7. Copy the LeadID column for anything with REMOVE into a new sheet
  8. Add a second column header called “Email”
  9. Save as CSV
  10. Open the Salesforce DataLoader
  11. Select Update
  12. Login
  13. Select Lead or Contact as appropriate and add your CSV file
  14. Create the field mapping
  15. Run that puppy
  16. (repeat steps 3-15 for Contacts)

Book review: The Dance of the Possible

Few categories of book have the potential to be as obnoxious as the “how to be creative” genre. Scott Berkun’s The Dance of the Possible fails to live up to its potential in that regard. In fact, it’s not really obnoxious at all. Instead it’s filled with a humorous approach to treating creativity as a skill to be honed instead of a magical epiphany bestowed from some mysterious muse.

Berkun goes out of his way to avoid giving an easy solution to being creative, and he’s very dismissive of creativity as an end goal. The point of being creative is to create something, and creativity as a virtue is a relatively recent development. Explore the possibilities of choices in mundane situations, he suggests. Somehow, this ended up with me writing on my sock with a permanent marker at 12:30 AM.


My oppressive pseudo-creative project. It will make sense if you read the book, I promise.

Any sort of creative work, even this very book review, is a delicate dance between two opposing forces: expanding what is possible for the project and contracting the scope so that it actually gets done. I can deconstruct the message of the chapters and reassemble them in any way I want, mixing them into something new. And I can shuffle these ideas around forever, but at some point the review must be published, or else what good has it done you?

One aspect of the book that I particularly liked was Berkun’s focus on some of the mental issues involved in trying to develop a creative work. In chapter 12, he talks about “the tightrope of creative confidence”: being confident enough to act, but not too confident. I prefer to think of it as “the eternal struggle between the Dunning-Krueger Effect and Impostor Syndrome”, but regardless of the name it’s a balancing act I know well.

All-in-all, The Dance of the Possible was a quick read. Indeed, the very first note I wrote down was “he seems to insist we not read the book.” This is not a book designed for Scott Berkun to wax poetic for chapters on end. Instead it shares real, actionable advice for exercising the thinking muscle. I enjoyed this book and found the framing of the problem and solution to be very helpful in understanding my own thought process. I can’t say that I came away with any sudden, brilliant insight, but maybe that’s the point.


The Dance of the Possible is published by Berkun Media. It goes on sale March 15, 2017. The author provided a review copy for this post.

Vulnerabilities in TSA PreCheck

Back in December, Bruce Schneier wrote about vulnerabilities in TSA PreCheck. His article leaned heavily on quotes from a former TSA administrator who felt that the PreCheck program should have stricter screening requirements. Schneier agreed with the vulnerability assessment, but drew the opposite conclusion. Since it has not been defeated, he argues, all screening should be reduced to that level.

That will never work.

I agree that the level of at-airport screening that PreCheck members get is sufficient. The question isn’t one of security, it’s of security theater. Having said that the current level of screening is needed, how can TSA officials say “nah, we don’t need that any more” and expect to keep their jobs? After all, terrorism is not a solved problem.

But this is also the longest stretch without a hijacking of a  domestic-origin flight since the 1960s. As Schneier points out, if there were terrorists who wanted to attack an airliner, you’d think they’d have figured out a way through by now. Considering the abysmal detection rate in internal tests, it’s not a stretch.

As I wrote on the blog post, the TSA has made for good theater. When I travel with PreCheck, I get essentially the same screening process that I did pre-9/11. But now it feels special. I get shorter lines and less hassle. But the lines don’t have to be long, and the hassle doesn’t have to be there.

Yes, TSA PreCheck has vulnerabilities. All systems do. It cannot be made fully secure, so maybe it should be retired. But really, it should become the standard.

Mythbuntu shuts down: is time up for time shifting?

The announcement of Mythbuntu’s death earlier this month got me thinking about the future of the DVR. Mythbuntu is/was a derivative of the Ubuntu Linux operating system that focused on providing a pre-packaged platform for the open source MythTV DVR system. The Mythbuntu team has apparently shrunk from 10 to 2, which is a pretty small team to manage an entire Linux distribution.

I am not surprised people left the project. People leave projects all the time. I’m also not surprised nobody stepped up. The time of the DVR is drawing to a rapid close.

I never used Mythbuntu, but I did briefly install MythTV on my home server a few years ago. My newborn’s sleep schedule prevented my wife and I from catching the shows we wanted to watch. I had some hardware issues, but MythTV itself was really easy to setup and use.

But shortly after I had everything configured, we canceled our cable TV. We just weren’t watching very much, and a Netflix subscription was much cheaper than a cable bill. Cord-cutting obviates the need for a DVR. But even people who keep their cable subscription often supplement with a streaming service. There’s just less of a need to record TV broadcasts anymore.

That’s not to say there aren’t advantages. I do not worry about something getting yanked from the streaming service if I record it. I can still watch locally-recorded shows if there’s a network outage. But for the most part, anything I’d want to watch I can get on demand without having to store it on my hard drive for a “just in case” that will never come.

I wonder how much longer DVRs will last. They are essentially VCRs without the magnetic tape, and VCRs are dead (Full disclosure: I still have and use a VCR). Open source and proprietary DVRs alike must find a new value proposition in order to survive. Maybe they can help me getting around to digitizing all of those home movies I have on VHS.

Book review: The Only Rule is it Has to Work

The instinct and tradition that go into making baseball decisions has given up some ground to statistics over the years. A sport that has long been obsessed with the most miniscule stat (if I heard “he has the best batting average after falling behind 0-2 in the first at bat of the second game of a double-header played on Thursday since Someguy Wholastplayedseventyyearsago” on a broadcast, I wouldn’t blink) is letting those stats drive some decisions. Billy Beane’s Oakland Athletics are the most well-known example, thanks to Moneyball, but it happens everywhere. Even in a small independent league in California.

The Only Rule is it Has to Work is the story of Ben Lindbergh and Sam Miller: two statheads who got to use a baseball team as a laboratory for testing their hypotheses about what makes a baseball team successful. While everything went right for the Sonoma Stompers at first, the season didn’t have the satisfying payoff that a fictional account would. But even in a four-team league, only one team gets to claim the championship.

In a sense, the ending helps make the book a truer representation of baseball. While every team (and every fan) wants that trophy at the end of the season, the road is not easy and more likely ends in frustration than the sweet joy of victory. But baseball can be a wonderful sport without that joy: just ask generations of Cubs fans who have lived and died without their team winning the World Series.

I had my first exposure to unaffiliated ball this summer when the Lafayette Aviators had their inaugural season a five minute bike ride from my house. I instant fell in love with “my” team: a group of guys trying to get just a little bit better while they’re in college, in the hopes that they’ll get drafted. Playing for almost no money, in front of crowds measured in the hundreds, these guys live a life so different than the big leaguers. Reading this book gave me a greater appreciation of what the players, managers, and owners go through.

Despite the frustrating way the season ended, Lindbergh and Miller must have liked what they had a chance to be a part of: they’re still listed on the Stompers’ website as Special Assistants to the General Manager. And I think that’s what this book is really about. The stats are nice, but like the players, Lindbergh and Miller just want to be a part of baseball.

Those of us who love the game can use this book to live vicariously. The authors aren’t superstar players, and they’re not even working in the big leagues. They’re just two passionate guys working with a team just above rec leagues. It’s not so hard to imagine yourself in their shoes, and for a moment you’re a part of baseball again.

Cell phone plans are changing

If you’re a cell phone plan geek (and certainly someone out there is, right?), last week was pretty interesting for you. First AT&T announced they’d be eliminating one plan and halving the data limit on a more expensive plan. Then T-Mobile followed up with their announcement of going to a single post-paid offering. This unlimited plan has some limits, which the EFF is looking into for a possible net neutrality complaint.

Moore’s law is an observation of the count of transistors on an integrated circuit over time, but it has been more broadly generalized to apply to many aspects of technology. Of particular note is the general trend of a technology to become significantly cheaper over time. This does not seem to be the case in the world of mobile phone service, which should be an immediate red flag for anti-consumer behavior.

I compared my current T-Mobile bill to my hypothetical bill under the new “T-Mobile One.” We pay $50/month for the unlimited talk/text plan, plus $20/month for my line (which includes unlimited data) and $15/month for my wife’s line (6 GB of data per month). The total bill before taxes and fees comes to $85/month. With T-Mobile One, we’d pay $120/month ($130 if we don’t autopay). This $35/month increase adds service that I’d pay just $5 more to get and also takes away the ability to use my phone as a WiFi hotspot (without being throttled to 2G speeds or paying an additional $3/GB).

I’ll admit that I don’t use my phone as a hotspot, in part because the coverage is questionable (or non-existent) in a lot of places that I might want to use it. But I’m already overpaying for data: my wife uses a few hundred megabytes a month and I average around 1 gigabyte or so. Only in July of this year when I was out of town for three of four weeks did I use more than 6 GB, and even then it was only 8 GB.

Perhaps if T-Mobile were going to put that extra money into expanding coverage, I’d be more inclined to go along with their plan. Instead, if I were to switch I’d get the same level of technology for a higher price. That’s not how this is supposed to work. It’s not clear at this point if existing customers will be able to keep their current plan. I assume in the short term that will be the case. If I’m forced to change at some point, I’ll have to go with a different carrier. If I’m going to get raked over the coals on price, I might as well get coverage.

DevOps is dead!

“$thing is dead!” is one of the more annoying memes in the world of technology. A Tech Crunch article back in April claimed that managed services (of cloud infrastructure) is the death knell of DevOps. I dislike the article for a variety of reasons, but I want to focus on why the core argument is bunk. Simply put: “the cloud” is not synonymous with “magical pixie dust.” Real hardware and software still exist in order to run these services.

Amazon Web Services (AWS) is the current undisputed leader in the infrastructure-as-a-service (IaaS) space. AWS probably underlies many of the services you use on a daily basis: Slack and Netflix are two prime examples. AWS offers dozens of services for computation, storage, and networking that roll out updates to datacenters across the globe many times a day. DevOps practices are what make that possible.

Oh, but the cloud means you don’t need your internal DevOps team! No. Shut up. “Why not simply teach all developers how to utilize the infrastructure tools in the cloud?” Because being able to spin up servers and being able to effectively manage and scale them are two entirely different concepts. It is true that cloud services can (not “must”!) take the “Ops” out of “DevOps” for development environments. But just as having access to WebMD doesn’t mean I’m going to perform my own surgery, being able to spin up resources doesn’t obviate the need for experienced management.

The author spoke of “managed services provider” as an apparent synonym for “IaaS provider”. He ignored what I think of as “managed services” which is a contracted team to manage a service for you. That’s what I believe to be the more realistic threat to internal DevOps teams. But it’s no different than any other outsourcing effort, and outsourcing is hardly a new concept.

At the end of the article, the author finally gets around to admitting that DevOps is a cultural paradigm, not a position or a particular brand of fairy dust. Cloud services don’t threaten DevOps, they make it easier than ever to practice. Anyone trying to convince you that DevOps is dead is just trying to get you to read their crappy article (yes, I have a well-tuned sense of irony, why do you ask?).

Product review: Divoom AuraBox

When Opensource.com hit 1 million page views back in March, Red Hat bought all of the community moderators a Divoom AuraBox as a token of its appreciation. I’ve been using mine for a little over a month now and I thought I’d share my impression with my ones of readers.

The back of the AuraBox, complete with special Opensource.com branding.

The back of the AuraBox, complete with special Opensource.com branding.

This is a fun toy. The primary function is a speaker (with both Bluetooth and audio jack support), a task it is pretty okay at. I mainly use it to listen to public radio podcasts while I’m washing dishes or baseball games while I’m doing work outside. I have used it for music some, and if your goal is to have sound come out, it works. PC Mag’s review took the AuraBox to task over the sound quality, but I think it’s acceptable. No, it’s not great (though I’m no audiophile), but it works.

The fun part comes in with the LED panel on the front. By itself, the AuraBox can show you the time, temperature, or some pretty patterns. When paired with a phone, the LED panel becomes your own. It will display notification icons for a few select apps (notably and annoyingly, that does not include Google Hangouts), but you can also create your own art. The app supports static or eight-frame animated images on the 10×10 grid. When I first unboxed it, my kids and I had a lot of fun coming up with new images to put on it. I spent more time than I should admit making a train animation. The grid size, frame count (and fixed rate), and limited color palette constrain what you can display, but it’s still fun.

The front of the AuraBox, displaying my attempt at recreating the Fedora logo.

The front of the AuraBox, displaying my attempt at recreating the Fedora logo.

The internal battery charges via a USB port, which means I can put either my phone or my AuraBox on the charger, whichever one seems to be lower. I haven’t tested the battery life, but I’ve done several hours of use without charging and it hasn’t died on me. The Bluetooth connection seems reliable, though it is a little jarring when you go just out of range and suddenly your pants are talking to you again.

My only complaint (and if anyone from Opensource.com is reading this, please don’t mistake this for a lack of gratitude!) is that it is a completely closed environment. I expect that from consumer technology, but given that it was a gift from a website focused on open source, I’d have hoped for more. If the box had a published API, I can think of several fun things to do.

Still, this is a useful device to have around. If sound quality is your primary focus, this isn’t the device for you. If you just want to listen to things and have a little bit of fun, then I recommend it.

Check your project for name collisions

In my article about NPM’s Kik fiasco last week, a commenter lead me to realize I left out an important point. The developer of the original Kik NPM module said he wasn’t aware of the messaging platform of the same name. That’s a shame, because the whole kerfuffle could have been avoided if he had done some cursory searching first.

In the era of startups that seem to be named after Scrabble trays, even if you’ve made up a name, there’s a decent chance someone already made it up. That doesn’t mean you can’t use it, but the context matters.

Avoiding collisions is important for two reasons: branding and legal. The branding consideration is less context-dependent, but is all about how users find and distinguish your project. When Opscode, the company behind the Chef configuration management tool, changed their name to Chef, it made searching for errors and documentation harder. “Opscode chef knife” had much more relevant search results than “chef knife”.

The legal issue is and is not context-dependent. It is in the sense that actually violating a trademark requires some likelihood that the public would be confused. For example, Payless Shoes and the PayLess grocery chain can coexist with a minimum of confusion. However, no actual infringement is required in order to threaten someone with a lawsuit. Most open source projects don’t have the resources to fight a lawsuit, so the natural reaction is to give in, even if the claim is baseless.

So what should you check for? A recent Opensource.com article has some suggestions. In addition, I’d look for collisions on social media platforms, especially if your project is related to that platform. Also consider how the name might be made into a logo. You don’t need one right away in many cases, but when it gets huge, you may want it.