It's ultimately up to you, and if the negative effect from dev really puts you off that much then go for raydric.
BUT...
You've already said yourself you mainly deal with physical attackers, so dev would definitely be best for that.
You want to lower incoming physical hits as much as you can so that your kaahi will outheal it. If you can do that then the extra damage from elemental attacks often won't make a huge difference anyway. Not to mention, as others have said, the cloak reduces some of those elemental attacks as well which lessens dev's negative effect.
About the expense bit... you're dealing with your first (maybe only?) SQI here. You want it to be as good as it can get for the way you play. Don't cut corners here, even if it will take a little longer to save up 
Oh, oops. I guess I didn't make it 100% clear. I'm pretty sure that whenever I get this, I'm going to go for Esma. But if I'm feeling lazy, I'd switch back to physical attacking.
Also, my Soul Linker is not my main character. I just liked that it could be both a physical and magical attacker (and I'm not even a fan of magical attackers, but I like Esma), and on top of that, the SQI sounds awesome. So I decided I'd go for it, but I'd take my time.
And you are correct in assuming that this is probably going to be the only SQI I try to make. Ha ha. (Everything else seems daunting. At least for me.)
So... let me get this straight. Although I'm sure Deviling's the best choice in both cases, you're saying that if I
was just a physical attacker, Deviling would be much better because I'd be hit more often than if I was using Esma (hopefully far away,
not getting hit). But if I'm mainly an Esma attacker, Raydric would be
okay since I (probably) wouldn't be hit by physical attacks. Is that a safe assumption?
I picked Noxious for WoE/PvP if you were curious, but it works great for me in PvM since most of the damage can be healed via kaahi 
Uh oh. Now that I'm understanding the whole magic vs. physical aspect, I'm unsure again. lol Noxious DOES sound nice now... So typically, you don't get hit by monsters that use close ranged attacks because you're far away, and for the monsters that CAN attack you with long ranged attacks, Noxious helps out with that. Does long range resistance include all properties, whether physical or magic?
its not switching. it's as you said. for thor, you need fire armor. for prison, bathory/ed/sleep/frozen.
you need to wear proper armor, whether you have immune or deviling.
and the 50% added damage is usually negligible with proper armor.
you're saying you're gonna spend 100m+ on a soul bound you-can-only-make-one item and not put the best card in it?
anyway, up to you \o.o/
I was using Thor's as an example. I really don't go there. Scares the hell out of me. For what
I do on RO, I've never really felt like I needed elemental armor (including going to Prison). For hopefully the last time,
I'm just a casual player. I'm well aware that there are better options out there, as is clear from me resisting the Deviling this whole time, but I just don't deal with them. I leave that stuff up to the uber players.