Left-click skill somehow changing

This only seems to happen with my fire warlock, and I cannot quite pinpoint why. I use left click for

Sigil Lethargy (weird I know), and sometimes I will load up a fresh run and the game has changed it to something else, such as

Apocalypse or or one of my other hotkey skills. I don't even have the Sigil bound to any hotkey or anything.

Now, I can sometimes force this to happen if I spam my hotkey skills while clicking. It seems that maybe if 2 skills are used on the same frame it changes which skill is on the left-click, or something like that. But as I said, this is happening when loading into a game as well, where I know I left the previous game with the correct configuration.

This is a small annoyance, I know, but it does sometimes lead to some sketchy moments when I'm trying to lay down a Sigil on a pack of baddies and instead it summons a

Tainted or something. I tend to not notice the skill has changed until I go and try to use it, but by then it's too late. And since it's an unbound (and left rather than right click for that matter) skill, I can't quickly mouse wheel to change it back, I gotta click on it and select it!

Just wondering if anyone knows why this happens or if I can prevent it somehow. I tried changing around where the hotkeys were, whether they appear on left or right click, and that didn't seem to matter.

Educated guess:
1) You are using quick cast
and 2) Your quick cast bind for the skill replacing the other is bound to the same side (left click vs. right click) as the skill being replaced

If so, that's a problem as old as the introduction of quick cast.

The cause for it is the simple fact that blizzard was too lazy and/or incompetent to actually implement a proper quick cast system so instead they implemented a shoddy, failure-prone band-aid solution.
Rather than actually just casting the skill when you hit the keybind for it, their crappy band-aid solution instead changes your skill to that skill (using the slot you bound it on), then casts the skill and then changes it back to the previous skill assigned to that button.
In addition to that just being a half-baked approach to begin with, they also didn't bother adding any proper error-catching to it.

As a result, if the cast execution fails for any one of the many possible corner case reasons (disconnect, hover target change to an invalid target, ...), the entire pseudo-quick-cast chain breaks down and stops executing fully. So as soon as that happens, the process just aborts and your skill is no longer changed back to what it was before it switched to the quick cast skill.

You can't really avoid that by any means other than (a) not using quick cast (ugh...) or (b) paying closer attention to how/where/on what you cast skills/what your mouse is hovering over so as to avoid execution failures to begin with. Though the latter will never be 100% reliable as something as simple as another player or friendly unit (merc/bound demon/...) moving underneath your mouse cursor can break it for you with no way for you to predict and hence avoid that.