SSD performance, not sure whats hapening

[colossk]

I bought an intel 320 SSD (120gb). When I import small files 30-60k hands I start off at about 700 hands/second and then the speed drops so at the end it shows me doing about 350 hands/second.

I returned the intel 320 and bought 2 vertex 2 SSD's. Again the same issue is occuring. When I import small files I get a starting hands/second import of 700-800 hands/second. Currently while I am writing this I am importing I million Stars hands. I have imported 901k hands so far of the 1 million. The initial Start speed was 825 hands/second, right now its sititgn at 51 hands/second with a runtime of 4 hour and 41 minutes and an esitmated 20 minutes remaining. Is this "normal"? Because it doesn't seem like it to me with all the posts I'm reading in the forums. Some peoplel have imported 1 million hands in 14 minutes 2 years ago.

When it goes thru the housekeeping phase(vacum, analyze and rebuild the cache), it's about 20x faster, it's just the importing that seems to be really slow.

I have read the topic How to make PT3 fly and changed the settings in there but it seems to make no difference.

ANy ideas?

[kraada]

Are you using PostgreSQL 9.0?

[colossk]

Yes 9.0.4 64 bit. After this post I just imported another 159k hands. The import took 10 minutes, started @ 680 hands per second and ended at 264 hands/second.

Does the size of the file have anything to do with the transfer speed?

I'm using windows 7 64 bit and I have an inted I7 Quad core 3.2 ghz, with 6 gigs ram if that helps any.

It seems that the larger the file that I import the slower the hands/second transfer rate drops as time drags on

[kraada]

We've been investigating similar behavior internally - I now suspect strongly that it has to do with PostgreSQL 9.0 - but I have yet to determine the cause. It's something we're going to continue to investigate to figure out what about 9.0 is causing this degradation over time.

[colossk]

So should I Downgrade postgres to an earlier version? Is that even possible?

[kraada]

We're investigating what we can do to fix things on our end; everything will work fine with 9.0, just I would recommend importing in small-ish chunks at this point in order to maximize speed (then do one big housekeeping run at the end).

[alexjcombs]

Kraada, I don't think this gradual slowdown in import speeds is specific to Postgres 9.0. I noticed the same problem when running 9.0, but the problem still exists after a fresh install of windows 7 and the use of Postgres 8.3. Processing imports in batches of 25k hands results in approximately 600 hps, whereas if I left a 3 million hand import alone, it would be chugging along at under 100 hps by the end.

[sonicman]

This has always happened for me whether ssd or ordinary hdd, any version of postgresql. Big imports always slow down the further i get into the import. Pt3 doesn't seem to utilise much memory to import the hands no matter how much you tweak the config file in postgresql. I have 8gb ram win7 64 bit and i can never get system to utilise more that 3gb in total
Maybe it's something to do with PT3 being a 32 bit program and i don't think they can utilise more than 4gb ram even though Pt3 is running on a 64 bit system and 64 bit postgresql. Hope Pt4 has 64 bit "extensions" so it can utilise the full potential of 64 bit systems and their increased memory limits!
Do you automatically run the autorate feature after manual imports as this always reduces the hand import speed, as it autorates after all the hands are imported but the hands/sec counter still keeps going (and reducing), maybe this should stop once all the hands have actually imported!
Kraada have you tried postgresql 9.0 32 bit and 64 bit to see if there is any difference? Maybe its something to do with how Pt3 handles 64bit systems and 64bit postgresql, but there again i just play poker, haven't a clue about software programming :)

[kraada]

I'll add that to my list to try - we've been working on improving the import speeds in general for PT4 and I think you'll be very happy with the results.