When and how should you decide to 'inline' a function?

The answer is the same as when to apply any other optimization: after a profiler tells you to, and not a minute sooner. There are a few valid exceptions to this guideline for cases in which you would reasonably inline right away, such as for an empty function that's likely to stay empty, or when you're absolutely forced to—for example, when writing a non-exported template.

Bottom line, inlining always costs
something, if only increased coupling, and you should never pay for something until you know you're going to turn a profit—that is, get something better in return.

"But I can always tell where the bottlenecks are," you may think. Don't worry, you're not alone. Most programmers think this at one time or another, but they're still wrong. Dead wrong. Programmers are notoriously poor guessers about where their code's true bottlenecks lie. Sometimes, we get lucky and are right. Most of the time, we're not.

Usually, only experimental evidence (a.k.a. profiling output) helps to tell you where the true hot spots are. Nine times out of ten, a programmer cannot identify the number-one hot-spot bottleneck in his or her code without some sort of profiling tool. In my years in this business, I've come across several programmers who have protested until they (or their colleagues) were blue in the face that this didn't apply to them, that they could consistently "feel" where the hot spots were in their code. In those same years, I've never seen such a claim turn out to be consistently true. We're good at fooling ourselves.

Finally, note another practical reason for this: Profilers are a lot less good at identifying which inlined functions should not be inlined.