Still seeing this, Rawr is practically unusable for me for my paladin at the moment.
Has anyone got any recommendations how to alleviate this?

I'm aware there's a related bug filed at Rawr's CodePlex site, but there's been no change to the situation for either Rawr2 or Rawr3 for the last weeks, so maybe I'm just doing something wrong?

You linked a crash issue, it is unrelated to any sort of performace problems in Rawr.

I haven't had much chance to do any kind of work to Rawr recently as I've been busy working on my new house I'll be moving to soon (tm) in my spare time.

Oops, yes, should have been this one, of course: Rawr - View Issue #17002: [Retribution][2.3.13] Very Slow to Optimize

Anybody know how to get newest version of rawr to show leather items?

I'm in a very similar boat to you. Basically looking at non-frost, non-raid gear and trinkets were a real bugger. Since my main spec was prot I dug around in my bags when I bought a second spec for dailies and ended up with chu chu's box of horrors and Crusader's Locket. After purchasing a few 100 EoT's worth of gear, including Mark of Supremacy, I really wanted ANYTHING and ended up farming up Banner of Victory with some friends. It was a significant upgrade to tell the truth. When swapping those two greens for Mark and Banner I pretty much shot up from 3k-ish to 4k-ish dps consistently. I believe the ArP may be weak but the 1008 AP procs pretty much right away then ICD seems to be about 45 seconds so this sucker lets you hit like a truck while doing trash pulls and early into boss fights especially if you get it to proc and throw up wings and kick off Mark.

Needle Encrusted Scorpion, Meteoric Whetstone, Incisor Fragment all score around the same, give or take, but you can only really farm up Banner and Scorpion. Almost all of these things have ArP anyway so you're just gonna end up with some anyway on that front. Thinking about grabbing a Mirror of Truth for bosses and such but really don't feel like trading down badges 160 times (40EoT->40EoC->40EoT->40EoH) just to have to remember to swap trinkets on a boss fight and make more work for myself until I find something better.

In fact, I might do that anyway, test a bit on a dummy and then post results for you, refund the thing and buy a Heirloom for my alt. If you need to compare gear and such, Brawlnstein, shadowsong.

TLDR. Farm it up, it's regular ToC so you can run it over and over in about 20 min and just throw out a NEED and you'll likely be very happy.

As a bit of a follow up, thank you to the folks that PM'ed me the "shift + Click" purchase functionality for emblems. So obvious, I would have never thought of it.

With that ease of purchase in mind, and the fact that I picked up a bunch of other junky blue trinkets running heroics, I'll probably test this out some time Saturday and post findings. I know it's not top tier raiding stuff but I figure there is a decent chunk of "seriously casual" folks like myself that run heroics and fine tune their toons but generally don't raid that could use the info.

Edit: Have completed testing for now!

I did testing on the Heroic Training dummy with recount as my meter. Durations were 3 minutes, you rarely see anything longer than 3 minutes at our level of play (IE: outside of raids). Trinkets, when applicable, were popped right away and then again at 2 minutes. Rotations were done using How>Judge>CS>DS>Cons>Exo in CLCret. Buffs were iBoM, 11/5/55 spec from EJ, running Retribution Aura and Seal was SoV. Latency is a very stable 33-35ms.

I was a QA tester IRL for a few years so my methodology and reproductions, etc, should be sound barring any obvious conventions I'm unaware of. For example, I hear that the heroic dummy may not be affected by ArP and there could be all kinds of other bad data I'm rolling into this so if someone knows a good faq for dps testing on dummies point me towards it and I can clean this up some more.

Trinkets tested were Banner of Victory, Mark of Supremacy, Mirror of Truth, Incisor Fragment, Meteorite Whetstone.

Quick summary: Mark of Supremacy, as long as you activate it, is easily the best of all of these without a doubt. After that the Incisor and the Whetstone perform amazingly well for Rares vs. the Mark and the Mirror but the Epics were more consistent and interchangeable. The best combo appeared to be Mark and Whetstone.

Results
Mark + Whetstone: Was able to peak at 3850+ DPS on the dummy, 3725.9 sustained.
Mark + Incisor: Peaked 3700+, sustained 3625.8.
Incisor + Whetstone: Just shy of 3700 peak, 3532.7 sustained. Truly the amazing ghetto option!
Mark + Mirror: 3650+ Peak, 3490.1 sustained. Best of the "double epic" solutions.
---Three way tie essentially follows
Banner + Mirror: 3600+ peak, 3557.3 sustained. Best "double epic" if you prefer procs.
Mirror + Whetstone: 3600+ peak, 3564.6 sustained. Similar to above but not epic ;-)
Banner + Mark: 3600+ peak, 3517.9 sustained. I was running this pre-test personally.
----The oddball tier
Mark + Incisor: 3600+ peak, 3572.6. This might look good but it only, suddenly, broke 3200 dps in the last 20 seconds or so of testing. Before that I thought it was going to be the worst for sure. Possible retest candidate.
----The lower tier
Mirror + Incisor: 3450+ peak, 3399.2 sustained. This the previous is one of the reasons I don't trust incisor.
Banner + Whetstone: 3450+peak, 3366.0 sustained. Worst solution for proc only I guess! They must really not cooperate well as the Whetstone is a great performer and Banner is middle of the road. Maybe some kind of ICD problem but I usually waited a minute or so while recording data and such between testing.

I saved rawr sheets before each test, I dunno if they would be helpful.

Just wanted to point out that I just downloaded rawr 2.3.17 today. I just noticed after I fixed the settings for ToJ it is showing a very high dps. I have done this everytime I download a new version and never had this problem and I double checked that I used the same settings as listed on the first page. Anyone else have this problem?

Edit- It appears I found the problem, once fixed it still shows toj doing several hundred more dps then before. Maybe this is correct I dont know.

Since 2.3.16 was taken down, I'm assuming you mean the changes listed here: Rawr

Using 2.3.17 I don't see a new trigger added for TaJ, it appears to be the same as it was in 2.3.15. Is the trigger internal, was it removed from .17 or what's going on here?

2.3.16 got pulled down because of some issue. 2.3.17 got released shortly after. One would assume everything changedin 2.3.16 is also in 2.3.17 and it seems to be for me.

I've taken a habit in refreshing all my items from wowhead after each install. Works for me.
Uses 'Melee Attack' trigger and not the 'Melee hit' other items are.

Is rawr assuming Shadowmourn using /cancelaura? What changes would I need to make to model it both ways?

I'm not sure if these are wholly responsible for the effect, but there are two things I can think of.

Firstly is that it may be one of those odd rotational cases where the spell global cooldown and other misc. effects mean that you get less abilities off because of the longer cooldown, resulting in lower DPS. I thought that there were efforts underway to minamise this effect, so that may not be it.

Secondly, RetRawr does not model (in my belief) mana consumption, which is a significant factor in the glyph's favour. With a shorter consecration cooldown you cast it more often, leading to higher mana consumption (which may or may not be an issue.)