Sunday, November 2, 2014

a java developer's interpretation of depression as one's own heap and garbage collector

this will be more personal than usual

Most of my posts are in the style of "here's how you accomplish x using y", but this post is going to be a significant departure. I, like many other people, suffer from depression; more so in the past few months than I have before in my life. I won't go into all the details as to why because not only are they irrelevant to the post but they're also not something I would dump on my audience.

Instead, I'm going to draw some similarities to the Java Virtual Machine, and how the heap and garbage collector, in my mind at least, closely resemble the ways I struggle to deal with depression.

if you have no idea what the heap, garbage collector, or JVM is...

Maybe you found this post by accident, but you were searching articles about depression. Maybe you use a different programming language. Maybe you're not a developer at all. In these cases, I'm going to give you an abstract, 10,000 foot view of what it is I'm associating to depression. If you already know these concepts, skip ahead to the next section. If you don't or want an entertaining analogy, read on.

If you indeed found this article because you're struggling with depression as well, then I'm sorry to hear that. Depression sucks, and it's crazy how debilitating something invisible to others can truly be.

I want you to imagine a house, filled with the typical things a house has, and a person living in it. To associate the technical ideas I'm going to delve into below, the house is the Java Virtual Machine, the person is a Java Application, and the things are the resources the person (or application) uses.

Some things in the house are quite large: sofas, beds, appliances. These are things people tend to keep for a long time, and are usually things a person uses frequently. Some things are small and get used and discarded frequently: food, packaging, disposable wares, etc; they're short lived possessions. A person's house is only so big, and eventually they can run out of space to put things. The more stuff you have, the longer it can take to clean your house as well. Bigger houses also take longer to clean (or are at least more expensive to clean; I suppose you could hire an army of cleaning people).

Some large items in your house may be things that offer little value that you decide to hold on to: that pool table you never use or that drum set you played a handful of times and abandoned after realizing the learning curve. Sometimes you buy a lot of new stuff that you plan on keeping around for a while, but there's too much large old stuff in the way to use it how you want to. It's even possible to get to a point where you don't really have room for new stuff at all and it's imperative you clear some stuff out, which can take serious time and effort.

All of this is analogous to how a Java Application runs: it's a person in a house with limited space, it needs some large things that are used frequently, but also takes in many small things that are consumed and discarded quickly or new large things that require space to exist. The problems are real too: you can run out of space and spend an awful lot of time trying to juggle what you have in a way that makes sense. In the examples below, the heap is the space you have, and the garbage collector is the process of clearing up space when you need to. In a Java application, the garbage collector will pause the application to reclaim those resources.

similarities to my own mind

I spent a fair amount of time thinking about this analogy, and it both amused and bothered me how much it fit the problem of depression.

I have plenty of experience troubleshooting Java applications that are performing poorly, often due to memory issues. Tell me what's happening, give me access to the server it's running on, and I can pull the problem apart and come up with what's wrong and how to fix it. At the risk of sounding pompous, as long as I have the right tools at my disposal, I'm really good at it too. Sometimes the application was holding on to large things that prevented new things from coming in. Sometimes the application had way too much disposable stuff coming in, or wasn't letting go of stuff it should dispose of. Whatever the case may be, these issues prevented the application from accomplishing what it needed to do, and I was able to identify the problem and correct the behavior once I understood the root cause.

Depression can have similar effects on my mind. There are large things that take up a significant amount of space, but really offer little value; they're a burden on the system and can prevent new things from coming in. Sometimes there's an abundance of short lived stimuli entering your mind, and all you can do is focus on those, even if they're noise: sort of like a mental DDoS. Some things are actually very disposable but you end up holding on to them because of an issue with how they're processed; I think of it almost like being a thought hoarder. Ultimately all of these things can slow you down to a crawl and prevent you from accomplishing the things you need to do.

These things are surprisingly similar in my mind, which ultimately begs the question: why can I see it when it's a process on a server and be completely blindsided and overwhelmed by it when it's happening in my own head?

resource contention, overload, and thrashing

There's a key similarity here in how the problems manifest: the resources of the system under duress are unavailable. The application isn't sorting things out itself: it may have a way of complaining that something is wrong via a failed health check or some error but can't adjust. Ultimately you're the one coming in, analyzing the problem, and figuring out how to adjust what's happening to make those resources available again. Meanwhile, the application is struggling and can't fulfill what it's designed to do; in extreme cases it becomes completely nonresponsive.

When you break it down, resources are being contended for and overloaded, which means they're not available as they should be for all the normal functions the application has to perform. Thrashing takes place when what little resources are left end up being consumed trying to alleviate the stress placed on the resources that are overburdened. This is what can cause things to come to a screeching halt for considerable periods of time.

Applications are lucky though; they can be restarted usually at little cost, and a restart can alleviate the problem at least temporarily. Not so in humans. I don't know what would even constitute the idea of a restart in that case. Moving to a new place and getting a new job? Getting baptized in a new church and going through some kind of metaphysical restart? Who knows...

Still though, a restart is a stop gap. Usually restarting an application only provides time before the problem comes back; the problem still hasn't been solved. The same can be true in us: you can do whatever it is you think is a "restart" but only delay the inevitable.

What can we do?

the application has to change

If you're talking about fixing an application, it means changing it. True, the literal Java application can't change itself, but I think you can see where I'm going with this.

The application needs to be able to manage its resources more effectively. The application has to be able to identify when considerable resources are being used for things that offer no value. The application has to understand when there's an abundance of noise coming in and try to discard it instead of hold on to it. The application has to be able to accept new things. The application needs to manage its resources in a way that allows it to perform what it's designed to do.

Sometimes small design tweaks can provide big gains. Some design changes are enormous and take time. In between these changes, the application still has to run though: it performs an important duty and can't just shut down during construction so to speak.

so what has to change?

I keep using terms like "large things that offer no value" as well as "lots of noisy stimuli" and I'd like to expand on that, at a personal level, to provide context. I'm going to emphasize that I'm not a therapist, I'm an engineer, and that everyone's problems are different, so this is purely an example and not some kind of empirical data.

I struggle a lot with the unknown. I dwell on the "what ifs" and "should haves" and "don't knows", struggling to reconcile what I should or could have done differently or how I need to prepare for them. Most of them I can't do anything about though; life will always do unexpected things that you can't be prepared for. I could get sick. I could be robbed. I could lose someone close to me. Can I prevent any of it? Well, there are some things I can do like eating well or having a security system. The more important question is: can I control any of it? The answer to that is simply: no. Despite that answer, I allow myself to be overwhelmed. I research compulsively trying to find answers to questions I can't hope to answer, and I consume significant amounts of time, and really as a resource time means my life, on doing so, when instead I could spend that resource on enjoyment. The end result of this process, for me, is depression. Not only do I become depressed contemplating all these things, but the realization of how much time and effort I spent in futility depresses me as well.

That struggle corresponds to a few of the analogies I've been using. For me, these thoughts accumulate as a significant amount of noise; to use the term I shared before they're a "DDoS on the brain." They're also large in scope and consume a lot of resources, so much that my brain stops focusing on other large things. What do I mean by that? Sometimes I forget to eat because I'm so absorbed in dwelling on the unknown. Sometimes I spend so much time on it I don't go to the store to buy food, or talk to my friends. Sometimes I lose so much time on something of so little value I either go to sleep late or stay awake thinking about it. I would consider all of those vital functions of myself as an application that I'm unable to perform due to a lack of resources.

the metaphysical gc pause

I don't always manage to do this, but sometimes I'm able to realize just how much of my resources I'm consuming on, well, garbage, and I have an opportunity to get back in control. For me, I equate this to a GC pause.

Like I mentioned above for those unfamiliar, the Java garbage collector will pause the application to reclaim resources so that the application can continue to run. To do this, it has to identify what should stay and what should go, and remove the things that should go. I try to go through the same process. I look at the state of my mind and think "what is currently circulating in here that's not helping me do what I need to do?" and try to purge it. Some of it is still strongly referenced and I can't quite get rid of it yet. A fair amount of it is new enough though, that I can clear it out. I'm not saying this is easy to do; it's only something I've started to do recently and I'm slowly getting better at it. What I am saying though is that it helps me to pause, examine the context of what's happening in my head, and truly identify the garbage so that my resources are available both for what I need to do to keep running and to take new things in.

so who's the person looking at the heap dumps and profiling things?

One of the points I made before is that, with an application, I'm the one figuring out what is wrong with it rather than myself, and how that makes a huge difference in my ability to understand what's wrong.

If we can do that for computer programs, why can't computer programs do the same for us? I have a plan for this, and that's why I'm asking for a small donation to start my work towards creating an algorithm to implement a cerebral garbage collector that...

I'm kidding. :) (but be honest, you were thinking "man, wouldn't that be nice!")

In all seriousness, talking to a therapist has been helping me tremendously. Doing so isn't an easy step to take; in fact I avoided it for a long time and can honestly say that in my case it was due to a bunch of irrational fears. It's important to find a therapist you click with as well; who seems to have a decent grasp on who you are as a person and can communicate with you beyond just words. I was lucky in that I knew someone who recommended a therapist who was perfect for me, and I know others may not be so lucky.

Having a therapist has been hugely beneficial because, to come up with another programming metaphor, I was able to hand over all of my runtime statistics to him, and he was able to identify the bottlenecks and resource contention that was causing me not to run properly. Gradually I've been able to adjust to account for this and have made improvements, but I know it's a long process, and that it's important to fix the bigger problems that require major design changes and not just rely on the small tweaks.

I should mention too that having a therapist doesn't imply taking medication. In my case it was the total opposite: I told my therapist from the start that medication was only on the table if I arrive at an impasse that I absolutely must get past but can't without a prescription, and he was completely open minded and empathetic towards my position.. That's not a one-size-fits-all situation for anyone, nor would I ever judge anyone for agreeing or disagreeing to medicate. I have my reasons for why that works for me; I wanted to call it out for those who may fear a therapist for this reason as I once did. You have choices in how you get better, and data will help you with those choices. The most important choice though is deciding you want to get better, and taking steps to get there.

the road ahead

I know the "life's too short" expression is something people have heard to death by now, but I wanted to call out it here. It's not worth suffering and making yourself miserable if you can avoid it. I plan on continuing to give my therapist the data he needs, to get feedback on where the resource contention is the worst, to figure out what data is garbage, and to tune my garbage collection to make the most use of the resources I have. I know that someday, like all other "applications", I will terminate (it feels like there's a joke there about developers being nicer to operations people). When I do, I want to be remembered as an application that, while it may never run perfectly, accomplished many great things and improved the health and function of the collective system overall. I don't want to be the application that doesn't produce much because its resources are tied up doing things that don't benefit it or anything else. I don't want to be the application that stops running, but others don't notice because it prevented itself from really standing out because it was its own worst enemy.

I guess what I'm trying to say is: I don't want to be Internet Explorer.

And no, this article wasn't just a massive run towards an Internet Explorer joke. It just felt good to end on a comedic note.

Thursday, October 30, 2014

how do I select message data from my iPhone backup?

don't worry, it's easy (promise)

Messages on the iPhone are stored in a sqllite file inside of the backup. If you have no idea what a sqllite file is, well, then you have a little more homework to do, namely install sqllite so you can... read a sqllite file. :)

Once you have that installed, you need to grab the file within the latest subdirectory you can find inside the directory Library/Application\ Support/MobileSync/Backup/ called 3d0d7e5fb2ce288813306e4d4636395e047a3d28. This file has your texts in it. I suggest you copy it somewhere else in case you corrupt it by mistake using sqllite. Once you have it, just run sqllite [your backup filename] to load the data.

getting the data

The following query will retrieve the texts stored in this file, in a nice pipe-delimited format:

If there's more data you're looking for, this will tell you the columns in the table (sqllite doesn't use desc or describe):

The output you get should be usable in Excel or any other number of tools if you want a different backup of your texts.

Monday, July 14, 2014

underscan problems using hdmi with your macbook pro retina? here's a potential fix

an answer buried in the middle of nowhere

Just went to hook up my monitors to my Macbook Pro Retina, one with DVI to Display Port, the other with HDMI, and my problem was immediately apparently: I was in underscan town...

Luckily I happened to stumble on an answer online, but it was a little obscure. Mind you, my monitor had the same option as the poster's monitor (a Samsung in my case) in the post below, so your results may vary, but it fixed the issue:

Okay, I was having the same problem... Trying to connect Macbook Pro to a 24in monitor... And I was SO disappointed I had to use the "underscanning" feature in the Displays box to make the bigger screen slightly smaller so it fit everything in there... This was horrible to me because it also made things less clear.


Go into your little settings within your ACTUAL MONITOR... And there's a section that talks about "Input"... Mine says "Input Select".. And where it mentions "HDMI", it gives an option of PC or AV... It was set to AV!! So I set it to PC and life became wonderful again! Spread the word. This seems to be a common problem but not everyone realizes it's the monitor.

The answer was found at I figured I'd toss a post up to try and make the solution a little more SEO relevant to those suffering from the same problem.

Saturday, May 31, 2014

caching in java: getting better performance and precision from your golden hammer


This post is actually a written form of a presentation I did at a corporate tech convention. The inspiration for it came from a common tendency for caching to be treated as a golden hammer for performance: if we were executing a database query frequently, we were caching it.

To be blunt, it's just not that simple. Sure, memory is relatively cheap, and it may make things go faster out the door, but that doesn't mean you're using your resources well. It also doesn't mean your caching approach makes sense in the long run either: how effective is your hit ratio when that data grows? Sometimes people don't think about acceptable levels of staleness in data either and just blindly cache for hours when it can negatively impact the experience of your application.

All of these data points mean there is no simple answer to caching. This isn't just a one-size-fits-all solution: you have to be precise and think about how your data is going to be used, how it will grow over time, and how stale it can get. To reference Idiocracy, I view caching as a toilet water solution. If you haven't seen the movie, here's the gist (spoilers ahead): 500 years in the future, a man from the past wakes up from cryostasis and is the smartest man in the world. He finds out the crops are dead (I won't say why) and suggests that people start watering them instead of what they were doing. In this dystopian future, people only associate water with toilets, so the response from everyone is "Like from the toilet?" After water solves the problem, discussion of other problems leads people to suggest "Hey, let's try putting toilet water on it!" The moral of the story is this: you can't blindly apply the same solution to every problem just because it solved one that sounds similar.

This post is designed to suggest questions to ask rather than provide answers for everything. At the end I'll present a checklist you can try out to see if it helps you.

what are the costs of caching in java?

Java memory management is a massive topic by itself, so we're going to only go skin deep here. If you're not familiar with how the heap is organized, here's a diagram (note: permanent generation is going away in Java 8 as a result of being collapsed into tenured):

We're not going to worry about eden versus survivor space in this post. What we're most interested in is the difference in young vs tenured, and how things end up getting stored over time. So...

  • The young generation stores short lived objects, and is garbage collected (or GC'd) very frequently. For an example of short lived objects, think of all the objects you create during a request in a web application that are unnecessary after the request is complete. Young generally uses far less space than the tenured generation, because most applications don't create so many short lived objects that you need a massive young generation.
  • As the garbage collector scans the objects in the young generation, if it sees objects that are still in use after checking several times, it will eventually move that object over to the tenured generation.
  • As your tenured generation has more data added, the amount of memory you use will grow to the maximum number you specified in your -Xmx setting for your JVM instance. Once you've reached that point, the garbage collector will collect from the tenured generation when necessary.
  • Different garbage collectors have different approaches when it comes to the following statement, but at a high level, once something has to move out of young generation and into tenured, if there isn't enough space in tenured, the garbage collector has to clear some. This means your application will suffer from some pausing, and this pausing will increase (sometimes dramatically) in either frequency, duration, or both with the size of your heap. The garbage collector has to rummage through the objects in the tenured generation, see if they're still being used by your application (strongly reachable), and remove them if they're not. Sometimes it has to shift the objects that remain (compacting vs non-compacting garbage collection) if there are several small blocks of memory free, but a single large object needs to move.
  • Generally, data you cached will live indefinitely in the tenured generation. I won't get into other cases here (though if you're interested check out my posts on other reference types and off-heap storage), because mainstream stuff like Guava Cache and ehcache will put stuff here.

So there you have it: ultimately the garbage collector is going to have to manage what you cache, which isn't necessarily a bad thing, but can become one if you're not careful. If you're going to force the garbage collector to do more work, you need to get value out of keeping that data in memory.

so what should we cache?

To the best of our ability: things that are frequently used and/or expensive to load. The former is the difference between a high and a low cache hit ratio, and we want a high cache hit ratio if possible. We need to know what data is accessed frequently or most expensive to load though in order to understand what to cache. Here are some criteria I encourage people to apply to their data:

  • Is there some data that is explicitly more popular? For example: if we're a retail website and there's a set of products we know we're going to promote in a highly visible place, that data is going to be explicitly popular because we're directing people to it.
  • Is there some data that is implicitly more popular? For example: if we're a social media site, we probably have a bias towards new content rather than old due to the way we present it. In that case, we can prioritize newer content since we know that, at least for some period of time, it will be popular until newer content replaces it.
  • If you were to graph traffic against some unique identifier in your data, where does the tail start? Often times the tail of the graph starts early on in the data set, and should give you an indication of what data is very valuable to cache, and what data will often be a cache miss if you were to cache it.
  • Is your cache distributed or local? If it's local, you're going to increase the cache misses to be equal to the number of hosts running that cache for any given entry. If it's distributed you may be able to cache more of the tail effectively.
  • Is the data expensive to load? If it is you may want to consider reloading that data asynchronously once it expires, rather than blocking on a reload. I've written more about blocking and non-blocking caches in another post.

Bear in mind the cost of loading data and preventative measures in terms of exposing yourself to a DDoS, or Distributed Denial of Service attack. Chances are you can't cache all of your data, and you should be careful to not put yourself in a position where cache misses and/or expensive load times can make your application unavailable if hit frequently enough. This subject is beyond the scope of this post, though I may write about it at some point in the future.

why will data live indefinitely if cached?

Interestingly, I had a discussion with a colleague about this just a few weeks ago, and it revealed a massive misconception that he and probably other people have about caching.

Usually, caches don't eagerly expire data by default; they do so lazily depending on their caching algorithm. Here's an example: say we have a cache that can hold 10 items, and that the max age is 30 minutes. If we load 10 items into the cache around the same time, and see how many items are in the cache 40 minutes later without having accessed them at all in between, we'll still have 10 items in the cache. By default, with technologies like ehcache and Guava Cache, there's no thread running in the background checking for expired entries. Since the entries are stored in such a way that they're strongly reachable, they're not eligible for garbage collection and will always stay in your tenured generation.

You may think this isn't a big deal, but it can be in practice. Let's say someone didn't pay attention when they set up a cache and decided it could hold 10,000 entries. Maybe the data set in the backing storage was small at first and no one saw a problem. Let's say that, over time, the data set grows to 10,000 entries, and that those entries are gradually hit over the runtime of the application, but only 100 or so are hit with any reasonable frequency. In that case, you're going to shove the entire data set in memory over time, and you're never going to get that memory back, even if no one accesses that data again for as long as the application runs.

Guava allows you to create an expiry thread that will make old data eligible for garbage collection (I haven't found the same in ehcache; comment if you're aware of this feature). Still though, you're probably better off sizing your cache more effectively for your traffic and data than you are eagerly evicting, depending on what problem you're trying to solve.

Bottom line: think about how much stuff you're tossing in memory, how much space it will take up, and how valuable it is to have it in memory.

wrapping up for now

This post is less of a deep dive into caching and more of a contextual overview of the impact of caching. I hope it helps clear up some confusion that I know I've run into multiple times. The goal of this post is really to identify why caching can actually go wrong over time, and why all of us really need to put thought into how we're using memory. The golden hammer principle can often apply to caching, and if you're not careful, you can really hurt yourself if you're swinging that hammer around a little too freely. :)

Wednesday, April 30, 2014

pro tip for developers: take some time to code up some well known data structures and algorithms yourself

remember all that stuff you learned in school?

It's probably stuff you take for granted when you use data structures built into higher level languages like Java, Scala and C# or libraries such as Apache Commons, Google Guava, and the like. Ultimately that stuff is the foundation of your knowledge as a developer: you don't think about it all the time but the fundamentals are ingrained in you.

That said, I bet if you cracked open a textbook today and decided to read about one of the classics, it may stump you for a bit. Do you remember the details of Dijkstra's algorithm? You may remember it's used for shortest paths in a graph, but you may not remember how it works. What about hash based data structures? Do you remember what makes a good hash function, or different strategies for handling hash collisions? How about string sorting or searching approaches?

If some of these aren't fresh in your head, you shouldn't feel bad. You may have a notion of how they work on a high level still, along with the computational and storage impacts, but you may not remember how to implement them offhand. Going back a few sentences, if you've ever reread these from a textbook, I'm willing to bet you didn't grasp all of it on the first pass. It's pretty dense material, and usually the best algorithms somehow cheat or have some type of trick that gets around some of the computational expense of the problem that you need to get your brain in sync with again.

Now you may be thinking "what's the point of all of this? I almost never have to be hands on with this in my day to day job, that's why it's not at the ready." This is probably the case, but it very well might not be if you're in an interview. High caliber companies will probably ask you about this kind of stuff, and may very well ask you indirectly as a word problem instead of saying "show me how a trie works."

why you should take the time to write some of this code yourself

Ultimately, we all learn a little bit differently, and we speak about problems in ways that only we understand. How much easier do you find your own code to read and comprehend versus someone else's?

For that reason, I'd recommend writing up some solutions yourself, with your own comments and flair, so that if you ever need to refer to it, you don't end up cracking open a big textbook and try to relearn from someone else's words. I had to do this in recent history for a task, we'll call it "homework", where having my own implementation helped me pull the solution out of the archived storage in my head and back into my brain's cache. Everything was familiar to me once I read my own code and my own comments; I even called out the things that tripped me up the first time I revisited the subject matter that tripped me up again when I looked at my textbook.

Big players will ask you this kind of stuff; Google is notorious for this and will typically suggest study guides to candidates, and there's also a famous blog post about how to prepare. I've known people who studied for months for Google interviews, and I'm willing to bet that they wrote out these algorithms themselves to prepare. So, given that, why not be ahead of the game?

stuff worth focusing on

Perhaps I've sold you on this idea. If I have, here's what I'd suggest for resources and problems to focus on to help you.

First, you should buy Algorithms by Robert Sedgewick and Kevin Wayne. It's one of the better algorithm books I've read (still dense, but reasonable), and has a lot of code samples and illustrations to help you understand.

Second, I'd recommend dabbling in the following as focus points:

  • Time-order complexity and Big-O notation (document the code you write with this, as fine grained as you can)
  • Sorting
  • Hashing and hash based data structures
  • Trees (binary/red-black)
  • Graphs and graph traversal
  • String sorting and searching
  • Data structures with interesting properties like min heaps and tries
  • Concurrency, for example map/reduce or a blocking work queue with a poison pill termination condition

All of these are pretty broad subjects, but they yield a lot in terms of the number of computer science problems related to them. Again, further emphasizing the point, writing code for the subjects above on your own gives you a persistent way to communicate to yourself in a very high fidelity capacity, and provides you some nice examples if you're tasked with having this knowledge at the ready.

I mean what I said about that textbook too; it's a very well written book that covers a wide variety of classic problems and algoritms, and you owe it to yourself to add it to your collection. :)

No matter what position you're in, I hope this advice helps you in your career in the long haul. If you have questions, or if you've done this and had it help you, please feel free to comment below!

Monday, March 31, 2014

using spring boot, spring websockets and stomp to create a browser chat service

when automagical becomes autotragical...

I have a love/hate relationship with frameworks that are automagical.

On one hand, they can save you a lot of time and effort, preventing you from writing tons of boilerplate code, and just less code in general (every line of code is one you have to support, right?).

There's a dark side to this however. Often times automagical frameworks have rules, usually modeled by conventions. If you violate these, the framework will slap you across the face with a stacktrace. Sometimes the stacktrace is meaningful. Sometimes it's outright esoteric. Sometimes it tells you something that you can't accurately discern without digging into the bowels of the code that you're trying to leverage without creating bowels yourself.

In this case, Spring was guilty of said crimes on several occasions.

Don't get me wrong; I'm quite fond of Spring. However, in this case the solution involved going roughly shoulder deep into Spring's bowels to find simple, yet not obvious solutions to problems. Here's a list of some of the problems I faced, in the event that someone out on the intarwebs is searching for a way to make sense of it all:

  • Spring Boot, at least when I wrote my code, seems to despise Tomcat 7.0.51 (though this appears to have been resolved now). This is particularly important, because in 7.0.51 they fixed a bug that prevented pure websockets from working correctly with... Spring (among other things no doubt).
  • When using Spring's WebSocket API, their scheduling API ends up hijacked. If you need to add your own scheduled bean, you need to override things (as shown in my code).
  • If you have a queue for your messages, and you don't have the URI start with 'queue', you can't use user specific message queues. Regular queues will work just fine.
  • The STOMP heartbeat configuration doesn't work for browser clients since you have to deterministically configure which clients you have that will need to support a heartbeat on the backend. I had to roll my own here.
  • You can't inject the SimpMessagingTemplate into your configuration class or it's bootstrapping path if you're using constructor injection. This can be important if you're trying to set up a ChannelInterceptor to filter out messages before they hit a resource in your application. (Also, how important is it to shave off the 'le' exactly? It really can't just be called SimpleMessagingTemplate???)
  • When I was using Spring 4.0.0.RELEASE, none of this was working correctly. Switching to 4.0.1.RELEASE magically fixed all my problems. I won't ask how or why.

There were several other issues that I've long since repressed. I wrote this code a while ago, and became so irritated with the number of gotchas that I didn't feel motivated at all to blog about it.

All of that aside, Spring's implementation is still leaps and bounds better than what Jetty is offering. For reasons I can't comprehend, the people who write the Jetty API seem to think that every class should be polymorphic to every other class, making it ridiculously confusing to set up. I cannot emphasize enough how many times during my use of their API that I thought "wait... this class is an instance of that?"

if you just want a working demo to play with

Then feel free to grab my code on Github. Did I mention that every example I found online was broken out of the box?

getting your configurations built

We're going to use four different features: web sockets, security, scheduling, and auto configuration. Each one will have its own configuration class, as shown below:

auto configuration, aka spring boot

security, because we need to log in to chat right?

scheduling, because we have a heartbeat to schedule with the web socket api

web sockets, because obviously

There are a few things worth pointing out in this configuration. The /chat and /activeUsers paths will correspond to Spring MVC controllers, while the /queue and /topic paths will correspond to the messaging API's pub/sub model.

designing the application

Before we get into the implementation, I'd like to address the design and talk about some of the details of using STOMP here.

First, you may be wondering what STOMP is. In this case, it's not an off-broadway production; it stands for Simple (or Streaming) Text Oriented Messaging Protocol. There's more about this on Wikipedia and the STOMP website, but in a nutshell it's a command based protocol for sending messages, interoperable with any provider or client implementing the STOMP spec. In the case of this example, it can use web sockets or not; it doesn't matter since the spec works either way.

STOMP and Spring allow us to set up queues, and more interestingly user specific queues. Semantically this is what you'd want in a chat application: each user can subscribe to a queue of messages coming to them, and the front end can route them into the correct chat window. In this case, our front end will simply subscribe to /user/queue/messages, and on the back end Spring will create a dynamic URL to map those messages to that matches the session of the user. The nice thing here is that each user subscribes to the same queue, but they only get their own messages. Even more sophisticated is that Spring can map multiple sessions for one user, so you can be signed in at multiple locations and get the same chat. All of this maps up with Spring Security as well, so as long as you have that configured all of your sessions will be mapped to the correct user queue, all messages received by the client will show up in a chat window corresponding to the other person in chat.

STOMP and Spring also allow us to set up topics, where every subscriber will receive the same message. This is going to be very useful for tracking active users. In the UI, each user subscribes to a topic that reports back which users are active, and in our example that topic will produce a message every 2 seconds. The client will reply to every message containing a list of users with its own heartbeat, which then updates the message being sent to other clients. If a client hasn't checked in for more than 5 seconds (i.e. missed two heartbeats), we consider them offline. This gives us near real time resolution of users being available to chat. Users will appear in a box on the left hand side of the screen, clicking on a name will pull up a chat window for them, and names with an envelope next to them have new messages.

writing an implementation

Since we're writing a chat client, we need some way of modeling a message, as shown below:

We also need a place to send messages to; in this case a controller just like you would create in Spring MVC:

There's some interesting stuff happening here. First, we're injecting an instance of SimpMessagingTemplate into our controller. This class is what allows us to send messages to individual user queues via the method convertAndSendToUser. In our controller we're also assigning the sender ourselves based on the session information that Spring Security has identified, as allowing the client to specify who the sender was produces the same problem the email spec currently has. It's worth noting here that the message is sent to both the sender and the recipient, indicating that the message has passed through the server before being seen in the sender's client's chat window. We ignore the case of sending a message to the recipient in case you're messaging yourself for some reason so that you don't get double messaging (though I should probably just ditch that case altogether).

Lastly, I'd like to draw your attention to the @MessageMapping annotation, which binds the method the annotation is on to the path we set up in our configuration class earlier.

That's actually all we need server side to send and receive messages believe it or not. Next is determining who's signed in.

We're going to start with a controller that receives a heartbeat from users who are signed into chat:

In this case, we're receiving a message that contains information in the header about who the user is, and mark their most recent heartbeat in a class called the ActiveUserService, which can be seen below:

When we call the mark method, we set an updated time for when the user last checked in, which is stored in a Google Guava LoadingCache instance. We also have a method that will aggregate the user names of all the active users, based on the metric of them checking in within the last 5 seconds. This is the information we want to send back to our topic, which is handled with the code below:

In this class, we combine the service and the template to send a message to a topic every two seconds, letting all clients know which users are available to chat. In return, users can reply when receiving a message, telling the server they're still active, creating a round trip heartbeat.

putting it all together with a user interface

And here's where I step out into the realm of things I actively try to avoid: creating a user interface. I am using jQuery here, which makes the task far easier, but I'm still admittedly not great in this department.

Before we go down that path, it's worth calling out that I used two JavaScript files that were present in the Spring guide for using their messaging API with websockets: sock.js and stomp.js. The first sets up the connection to the server, while the second implements the STOMP spec. They're both available for download (and originally found) at Github.

Warning and full disclosure: I'm a lousy JavaScript developer. The code below is likely horribly inefficient and could probably be rewritten by a UI developer in about 10 lines. That said, it does work, and that's good enough for me for now. If anyone reading this would like to advise me on improvements I'm open to feedback. I'll probably try to clean it up a bit over time.


Before we can do anything, we need to connect (this fires when the document is ready):

In this method, we do three very important things:

  1. We capture the user name for the session (this is used for figuring out who we're chatting with and for coloring text in the chat window).
  2. We subscribe to a queue that's bound to our user, and call showMessage when we receive one.
  3. We subscribe to a topic that sends out all the active user names, and call showActive when we get an update.

There's something slightly peculiar here worth calling out: notice that we construct the SockJS instance with /chat as the URL, yet you may notice that the other JavaScript below sends messages to /app/chat. Spring seems to have some kind of handshake here that I don't fully understand. If you change the value in this constructor, it will fail saying that /app/chat/info doesn't exist, so Spring appears to expose the resource /chat/info here to facilitate a handshake with SockJS.

showing a message

Whenever we receive a message, we want to render it in the chat window for that user, which is done in the code below:

First, we figure out which chat window we should load based on comparing the recipient to our username. We then create a span with the new message, colored to indicate the sender, append it to the window, and scroll to the bottom of the window to show the latest message. If we're updating a chat window that's not currently visible to the user, we render an envelope character next to their name in the user list to indicate that there are pending messages. Below is the code to create the envelope:

getting the chat window

In the example above we obtain a chat window to update with message, but we have to create it if it doesn't already exist, as shown below:

We create a div to wrap everything, create a div for displaying message, and then create a textarea for typing in new messages. We automatically hide created chat windows if another one already exists, because otherwise we'd interrupt the user's current chat with another person. We create binds for two events: hitting 'enter' and clicking 'Submit'. In both cases, since the event references a DOM object that has a unique id referencing the user you want to send the message to, I'm capturing that and using it to route. JavaScript's variable scoping is something I often mix up in practice, so I'm relying on the DOM to tell me who to send the message to instead. The method for doing this is shown below:

We send the message to /app/chat, which routes to our MessageController class, which will then produce a message to the appropriate queues. After sending the message, we clear our the textarea for the next message.

showing active users

As I mentioned above, whenever the server tells us about the active users, the client reports back that it's active, allowing us to have a heartbeat. The first method in the code below sends the message back to the server, while the second renders the active users:

There's a lot going on here, so let me break it down:

  • First, capture who was previously selected.
  • Second, capture which users had pending messages. We'll want to re-render this with the new list.
  • Create a new div for the next user list.
  • As we process users, preserve the one that was previously selected by assigning the appropriate CSS class (.user-selected)
  • Bind a click event to each user that will hide the current chat window, removed the select class from all users and show them as unselected, remove the pending messages status, pull up the chat window for that user, and mark that user as selected.
  • If a user was added to the new list that previously had messages pending, redisplay the envelope icon.

There's really almost no HTML to this example; virtually everything is generated as messages come in. I opted for a lot of re-rendering of data to avoid an overabundance of conditional checking, though I don't know if this is good in practice or not: again, I'm not particularly good with JavaScript

try it out!

Like I mentioned, this is available on Github for you to play with. I know there's a lot covered in this blog post, and I'll probably update it several times to fill in gaps as a read it and/or get feedback. I hope this helps you get started using Spring and STOMP for doing some pretty sophisticated messaging between the browser and the server.

Friday, February 28, 2014

using jackson mixins and modules to fix serialization issues on external classes

the problem

Let's say you have some classes coming from another library that you need to serialize into JSON using Jackson. You can't manipulate the source code of these classes one reason or another, and you have a problem:

These classes don't serialize correctly

There are a number of reasons this can happen, but in this post we're going to focus on two examples: two classes that have a recursive relationship to one another, and a class that doesn't conform to the bean spec.

dealing with recursive relationships

Let's say we have two classes, User and Thing, as shown below. User has a one to many relationship with Thing, and Thing has a many to one relationship back to its parent, User:

Given these classes, let's say in a unit test we create users using the following code, establishing the recursive relationship:

Now that we can create a user, let's try serializing:

If you run that method, your test will fail, and you'll see a nice CPU spike on your computer because this just happened:

fixing recursive relationship issues with mixins

As it turns out, Jackson has an awesome feature called mixins that can address this type of problem (remember, we're assuming User and Thing are not modifiable).

Mixins allow you to create another class that has additional Jackson annotations for special serialization handling, and Jackson will allow you to map that class to the class you want the annotations to apply to. Let's create a mixin for Thing that specifies a @JsonFilter:

Now you might be thinking, "What's that 'thing filter' string referencing?" We have to add this mixin to the object mapper, binding it to the Thing class, and then we have to create a filter called "thing filter" that exludes Thing's field of user, as shown in the test below:

If you run this test, you'll see that it passes.

dealing with a class that doesn't conform to the bean spec

Let's say we have two other classes, Widget and WidgetName. For some reason, the person who wrote WidgetName decided to not conform to the bean spec, meaning when we serialize an instance of Widget, we can't see data in WidgetName:

Let's say we're creating widgets using the code below:

If we try to create a widget like this and serialize it, we won't see the name. Here's a test that will serialize:

And here's the output:

fixing classes that don't conform to the bean spec with modules and customer serializers

We can address this problem pretty easily using a Jackson module that provides a custom serializer for WidgetName. The serializer can be seen below, and it uses the JsonGenerator instance to write the value from the WidgetName argument:

We now need to wire this up in order to use it. Below is a test that creates a SimpleModule instance, wires up our custom serializer to it, and registers the module within our ObjectMapper instance:

If you run this test, you can see the correct output for the serialization:

conclusion and resources

All the resources used in this example can be found on Github at If you end up having to serialize data that's outside of your control and is causing you problems, you should be able to get a lot of mileage out of these two solutions.