As this change happened, I was very disappointed and sad

You are never going to reach any place in your life where everything is "perfect" forever. It defies the way that this physical reality operates.

Eternal change is the promise of this physical reality. Fresh, new desires for something better are always going to keep coming to you because the launching of those fresh, new desires is what draws life-force through you and keeps you energized.

You can never not have new desire which is why all those philosophical systems based upon the idea of "desirelessness" are ultimately doomed to failure while those people still want to stay alive...because that ultimate desire for survival (and everything that stems from it) still runs rampant through their lives. Just stick a pillow over their faces to stop them breathing for a few minutes and judge for yourself how much "desirelessness" they have truly mastered :)

So, because you are never not going to have new desire, you have to find a way to live that makes you comfortable with the idea of eternal change, because each fresh, new desire is going to lead to a change of some kind.

What you've got here with your boss-change is just the launching of a new set of fresh desires. Any meaning you apply to why it happened is illusionary anyway, it only matters now what your response to it is.

And the most empowering response is always to...

"Embrace Change"

In truth, you have no choice. You either embrace change or change is going to embrace you, whether you like it or not :)

And if you truly get yourself into the idea of embracing change, you develop a sense of invincibility that makes fresh, new desire something to welcomed in your life, not something to feel concerned about.

With that attitude, you develop a sense of adventure about your life. I guess you could say you develop a Star-Trek-ish Boldly Go kind of approach to living :)

So, having said all of the above, here is my one line answer to your question...

How can one tell the difference between unwanted incidents caused by sloppy/negative thinking and unwanted incidents that occur to help us learn and evolve?

...It doesn't matter one bit :)

FOOTNOTE: Some years ago, I asked Abraham a similar "Why did this happen?" style of question and I've written about what they told me in Despite feeling good most of the time, I have huge debts and can't find a job. Where am I going wrong?. That story might give you a few further insights.

EDIT - JAN 13, 2014

In response to your edited question in which you've added...

I have a different opinion on the "it doesn't matter one bit" though. I will try to illustrate why in this way:

If my unhappiness caused by my new boss is caused by sloppy, negative thinking, then I have more unpleasantness coming my way. Next, I might be assigned to work with colleagues with unpleasant personalities and I might run into negative and difficult students (I am a teacher) more often. And by the end of the year, I will have been in a negative state of mind so often that my health might begin to suffer. And my family life will suffer.

If my unhappiness caused by my new boss is a blessing in disguise which has not occurred by sloppy, negative thinking, then it is not likely that a series of other unpleasant encounters with other people will occur, which will free up my energy to make the most of this as a life lesson which in turn will make me grow and evolve and attract wonderful people. And by the end of the year I will be in good health and have a wonderful family life!

To answer your insistence that "cause matters", consider a car satellite navigation system.

If you look at the picture there, can you see on the display where it says, not where you are going or where you currently are, but where you started your journey from?

No, neither can I :)

Satellite navigation systems don't care where you have come from, only where you are going.

Your emotional guidance system is remarkably similar to a satellite navigation system. It will get you to your destination if you follow the guidance of it.

With a Sat Nav system, the guidance comes in the form of arrows on maps to follow. With your emotional guidance system, the guidance comes in the form of positive and negative emotions.

In both cases....It doesn't matter where you came from, it only matters that you now follow the guidance

If you follow the path of feeling good in whatever you do, you'll always end up where you want to go...and that's towards more things in your life that feel good. A happy route will lead to a happy destination, an unhappy route will lead to an unhappy destination.

I think what might be confusing you regarding these ideas is that you appear to have bought into the idea that the purpose of our physical lives is growth/evolution/learning (or whatever) on the way to getting your "physical stuff" from which place you can then live a happy life. And I would say that's not how it is at all :)

Growth, in the spiritual sense, is a "Bigger Picture" by-product that is dealt with at the Higher Self level and you may never even be consciously aware of what that "Bigger Picture" is...and it doesn't matter either whether you are consciously aware of it or not.

The purpose of your physical life is to Seek & Experience Joy...Eternally. Period.

Whatever illusionary things appear to get in the way of your seeking of joy is stuff you've set up for yourself at some level to make the journey worthwhile....because the joy is to be found in the journey itself, not in reaching the ultimate destination. The destination is just the excuse for the journey. See Ithaca.

Or, to put it more metaphysically, the obstacles you create for yourself are just excuses to align your physical focus with your ever-expanding Higher Self in good-feeling ways. See What is the point of seeking material wealth?

I've explained more about many of these ideas in Is life really just about living in a state of bliss?

From a spiritual perspective, this is going to be so enriching for you.

You actually have been given a brand new blessing. The way you have worded your question I know exactly what the go is:

You had an old boss you liked. You had an amazing experience with them.

The old boss is gone, a new boss, whom you don't like or raises challenges, is here.

And that is why your new boss has gone. Because you have evolved spiritually, you are ready for the next level. Welcome your new boss.

It makes no sense for your old boss to keep on lingering around. You have a great time with him, things are always blossoming. Your new boss is your new source of teaching. You are going to learn so much more from him than your old boss, and eventually, over time, you may end up becoming best friends with him and you will be sadened when he leaves!!

I have witnessed miracles in my life, of my worst enemies becoming very good friends of mine. With enough compassion and love, with real unconditional love, you can literally change your boss.

It is not easy. Few people can do it.

Yet I am willing to have a wager with you, that with persistance of your loving intentions, things will change.

Spiritual understandings of current events cannot be realised until the future. Until then, just you watch how this all will lead to good things!

In reality there is no difference because everything and everyone we encounter brings lessons that help us to learn-- either about ourselves or others. If we take the opportunity to receive these encounters with the aspiration to learn then of course inner growth or evolution will be the end result.

Sending you harmonious energy for all of your constructive and harmonious endeavors. Love and Harmony

In larger perspective, there is nothing called unwanted/desirable incidents. there are only incidents: One thing leads to another. What appears to be negative now may lead to some great surprise later, as exemplified in the following historical incidents:

A farmer who was working in a farm heard a cry of a boy fallen into a deadly marshy land. He rescued the boy risking his own life. The boy was happened to be the son of a rich man, who wanted to reward the farmer. Yet farmer refused to take any reward. Hence the rich man decided to sponsor the study of the child of that poor farmer. Thus the farmer's child became the world famous scientist Alexander Fleming who invented Penicillin.

Now see the intricate ways of Karma. The boy whom the farmer rescued was Winston Churchil who became the UK PM and successfully led the World War. The first use of Penicillin was successfully made on whom? On Churchil's father who sponsored Flemings's studies.