Tuesday, March 14. 2006Serving Images from a File SystemComments
Display comments as
(Linear | Threaded)
I'm glad someone's actually trying! And indeed, our problem is when the cache is empty, and performance of the site suffers greatly. As we expand more and more, that 36 million number will only grow larger -- we project by the end of 2007 that number will be 330 million. Which means the cache won't always be hit (sure we can add memory)....etc.
We're sacrificing small bits of performance, this is true. It's not something I'm denying. But we're sacrificing a bit of performance always, to maintain consistent performance. ie, a 5% performance hit to make sure that those days when the filesystem cache dies that it doesn't slow down the entire site. Not that anyone asked. Seeing folks scream "BUT YOU'LL TAKE A PERFORMANCE HIT!" is amusing, but unhelpful. In the long run, it's better. For us. In this narrow case. I'd like to see a comparison between retrieval time from disk versus retrieval time from db. overhead, yes, but it's low enough that we can live with it! QUOTE: I'd like to see a comparison between retrieval time from disk versus retrieval time from db. overhead, yes, but it's low enough that we can live with it! I'm afraid my statistics went in the recycle bin once I satisfied myself of the results, but I recall seeing between 0 and a 10% hit for using the database to serve smallish images. I took five measurements, threw out the outliers, and averaged the middle three. Still, some trials showed database performance hits below the "noise level" -- the variations between individual measurements were greater than the average difference between file system and database. I decided that, in my particular case, the many advantages to storing the images in the database far outweighed the relatively small speed impact. So many other things have bigger impact. (Having an image much bigger than the display size, for example -- I pre-scale images to match display size when I store them in the database in three sizes.) The funny thing about "conventional wisdom" is that it generally respects only other people's conventions. You can "prove" anything you want on paper!
i agree, and dont forget the update case. Deleting files and replacing them with bigger severly will result in fragmentation. (However growing files are much much worse than a few blocks variation in images)
One advantage of db backed up files (besides searching for attributes, which can also be solved with a hybrid solution) is backup/replication. Some filesystems may also degenerate much faster, especially if the main memory of the system not only can be used for dcache. Me thinks a fair compariion would compare a database server against an NFS server or cluster filesystem. I guess nobody would want to scale-up in an web-server scenario. Gruss Bernd
For the sake of backup and clustering, I still don't get why people scream when you save images to a database.
Point taken that it's maybe not as fast (and so on), then serving directly from a filesystem (using a small httpd), yet, the features (clustering, backup) easily outrule the loss of performance you encounter. Benchmarks are one thing, real-life experiences another. The TOC on buying another node outrules tweaking servers for hours and hours, and hours because you gain too little. I am not saying that it is sometimes not useful to do this, or one should not write effecient code. Not at all! But this is along the lines of writing hard to maintain code/applications for the sake of microseconds here and there. |
QuicksearchComments
about Wed, 31.10.2012 13:46
My friendly Offtopic Channel
in IRC has horror stories
about this, it is Halloween,
after all.
"Well, I do
have [...]
about Mon, 15.10.2012 09:41
What if we use SSDs. Can we
then assume memory limits
are not as significant
because SSDs can handle
random access [...]
about Fri, 05.10.2012 07:46
In fact, I am pretty certain
that this also breaks
mysql_upgrade or has the
potential to do so.
about Fri, 05.10.2012 06:44
It is quite a bit worse than
that: Right now, the default
RPM as distributed by Oracle
will fail to install if it
[...]
about Thu, 04.10.2012 18:52
> MyISAM in mysql.* has to go
before GTID can be
sucessfully deployed.
I
think you meant to say
before it can b [...]
about Thu, 04.10.2012 18:36
Sorry, it wasn't clear given
that none of this is
actually mentioned.
about Thu, 04.10.2012 13:47
You can turn it on with FLUSH
PRIVILEGES once the tables
exist.
about Thu, 04.10.2012 11:34
I think I have to install a
comment voting plugin just
so that i can +1 what Justin
just wrote.
Categories
Blog AdministrationDisclaimerAll examples, code-snippets, advice, trivia, etc. are supplied in the hope that they are useful, but no warranty of any kind is given. Use of trademarks is not intended as a challenge to them.
ImpressumCopyright 2005-2012 bei den jeweiligen Autorinnen der Beiträge. Die Infrastruktur von MySQL-dump wird betrieben von:
Azundris, Immanuelkirchstraße 18, 10405 Berlin, Germany
|
In Semi-Dynamic Data, Sheeri writes about Semi-Dynamic Data and content pregeneration. In her article, she suggests that for rarely changing data it is often adviseable to precompute the result pages and store them as static content. Sheeri is right: Noth
Tracked: Aug 17, 19:06