Matthew Price

Golang: Missing or Incomplete Socket Messages

I recently ran into a puzzling error while working with Go socket connections. When performing a read operation, incoming messages would periodically arrive incomplete or not at all. The fix turned out to be easy, but the problem wasn’t immediately obvious to me.

Almost every Go server implementation is built on top of the io.Reader interface. Go also provides us bufio.Reader, which implements buffering for an io.Reader object. It’s common to run into the following pattern for reading bytes from a connection:

data, err := bufio.NewReader(conn).ReadBytes('\r')
if err != nil {
    // Handle error
}

But we rarely need to read just one message from a socket connection. To continuously wait for new messages, we can enclose our code in a for loop and hand off processing to another goroutine. If we’re feeling clever, we can reduce memory usage and processing time by using a pointer to pass off our data:

for {
    data, err := bufio.NewReader(conn).ReadBytes('\r')
    if err != nil {
        // Handle error
    }

    go processData(&data)
}

Everything looks good so far, but this code actually introduces a big problem that you might not notice during testing: we’re creating a new bufio.Reader, and underlying buffer, on every iteration. Since the buffer is not persistent across iterations, any messages received before the new Reader is created will be lost.

The proper way to write this code is to create a new Reader outside of the for loop. You can then call its read methods like normal on each loop iteration:

reader := bufio.NewReader(conn)

for {
    data, err := reader.ReadBytes('\r')
    if err != nil {
        // Handle error
    }

    go processData(&data)
}

Survivorship Bias ➔

David McRaney, tackling misconceptions about studying success:

The problem here is that you rarely take away from these inspirational figures advice on what not to do, on what you should avoid, and that’s because they don’t know. Information like that is lost along with the people who don’t make it out of bad situations or who don’t make it on the cover of business magazines – people who don’t get invited to speak at graduations and commencements and inaugurations.

This is a long read, but it’s particularly worthwhile for recent college graduates who are slowly making their way into the world. It’s easy to furiously soak up advice from people and companies we admire, but we often forget how much we can learn from failure—including our own.

Printing TODO Comments with Ack

Example TODO Output

There are a lot of terminal commands floating around that will list the TODO comments in your source code, but I’ve always found their output format lacking. Last week I decided to come up with something better. It’s language agnostic and it organizes everything into vertical columns to make reading easier:

ack -i \
    -o \
    --group \
    --color \
    --sort-files "\b(TODO|FIX(ME)?|OPTIMIZE|BUG)(\(\w+\))?: (.*)" \
    --ignore-dir={.git,node_modules,vendor,Pods} \
    | perl -pe "s/:/\t/" \
    | perl -pe "s/\t(\w+)(\(\w+\))?:(.*)/\t\$1:\$3 \$2/"

Here’s a breakdown of the options we’re using, and a few others you might find useful:

Option Description
--color Highlight the matching text
-follow Follow symbolic links
--group Group matches by filename
-i Ignore the case of matches
-n Don’t search within subdirectories
-o Display only the matched text1
--sort-files Sort the outputted files in alphabetical order
--ignore-dir A list of directories that should be ignored

Updates

  • Fixed a bug where words such as “suffix” were captured, and added examples for ignoring directories. — 17 April 2016
  • Added support files whose names begin with a period. — 30 July 2016
  • Moved Go-style attribution to the end of the comment. This should make the list easier to scan. For example, “XYZ(who): Comment text” is now displayed as “XYZ: Comment text (who)”. — 25 May 2014
  • Added support for “BUG” and Go-style attributed comments such as “BUG(who)”. — 29 April 2014
  1. When combined with wildcard matching, this helps us discard any tabs or spaces that come before the comment. 

Sochi 2014: The Opening Ceremony ➔

Fireworks Over Fisht Olympic Stadium

As usual, The Atlantic and Boston.com have posted beautiful photo essays, this time for the opening ceremony of the 2014 Winter Olympics in Sochi, Russia.

The scale of their opening ceremony is astonishing: the floor alone required 120 projectors!

Merry Christmas

Christmas Card 2013

Wishing you all a merry Christmas and a happy New Year.

Carriers Reject a Plan for Preventing Cellphone Theft ➔

Brian Chen, reporting for the New York Times:

The emails […] suggest that the carriers are concerned that the software would eat into the profit they make from the insurance programs many consumers buy to cover lost or stolen phones.

Was there ever any doubt that carriers are more interested in profit than helping consumers?

How to Blog about Code ➔

Garann Means:

Our communities need to hear from people who aren’t the maintainers and conference speakers and web celebrities. We need to hear from people who give zero f***s, who never worry about their Klout scores or how many people starred their repo. The big names create an echo chamber where ideas are safe and popular and failure and being wrong are covered up so no one else can learn from them. We don’t really badly need any more of that crap. We need you.

On Apple and the New iPhones ➔

Victor Agreda, Jr.:

Apple isn’t innovating. Apple is just iterating. Well, yes. But do we worry Mercedes will disappear because the company has yet to introduce a flying car? What’s the last category-redefining washing machine you couldn’t wait to try? And yet people still buy them — the mind boggles.

All the Apps Have Been Written ➔

Kevin Hoctor, on the importance of creating things you’re passionate about:

It took ten months to design, acquire funding, and ship the 1.0 release. In those months, there were dozens of episodes where I felt too stupid and lacking the Cocoa skills I needed to get MoneyWell completed. I’m sure my wife was exhausted in her roles as cheerleader and therapist. It’s a damn good thing that I made this personal.

Movie Review of The Isle ➔

Roger Ebert:

Most people choose movies that provide exactly what they expect, and tell them things they already know. Others are more curious. We are put on this planet only once, and to limit ourselves to the familiar is a crime against our minds.