Snitz Forums 2000
Snitz Forums 2000
Home | Profile | Register | Active Topics | Members | Search | FAQ
Username:
Password:
Save Password
Forgot your Password?

 All Forums
 Community Forums
 Code Support: ASP.NET (Non-Forum Related)
 ASP.NET Conversion
 New Topic  Topic Locked
 Printer Friendly
Previous Page | Next Page
Author Previous Topic Topic Next Topic
Page: of 11

skwayb
Starting Member

17 Posts

Posted - 03 November 2006 :  15:26:28  Show Profile
Can't wait to see this... Any ideas of when you will have something we can play with and help test?
Go to Top of Page

HuwR
Forum Admin

United Kingdom
20550 Posts

Posted - 03 November 2006 :  15:40:20  Show Profile  Visit HuwR's Homepage
It is getting closer I don't want to give a date yet as it would put me under pressure to try to get it done, and I am busy doing "real work" at the moment so am not able to do as much with it currently as I would like, but it will definitely be before the end of the year, that is about as firm as I can be at present, however initially it will be for online testing only, it will be some time before it is ready for others to install and test.
Go to Top of Page

wildfiction
Junior Member

167 Posts

Posted - 03 November 2006 :  22:16:14  Show Profile  Visit wildfiction's Homepage
All your hard work is much appreciated Huw!
Go to Top of Page

StephenD
Senior Member

Australia
1044 Posts

Posted - 11 December 2006 :  04:08:41  Show Profile  Send StephenD a Yahoo! Message
Looking forward to this Huwr!
Go to Top of Page

HuwR
Forum Admin

United Kingdom
20550 Posts

Posted - 11 December 2006 :  04:58:41  Show Profile  Visit HuwR's Homepage
Update:

Not had much time to do anything recently (need to earn some money ) however I am hoping to get a test site up ready for testing/preview in the new year, no promises, as real work is quite hectic currently, but that is plan if all goes well.

It will not be available for you to download at that time as it still has quite a bit of work that needs doing, but would like to get what I have done upto that point fully tested before continuing.
Go to Top of Page

janaf
Starting Member

Sweden
19 Posts

Posted - 13 December 2006 :  10:13:51  Show Profile
Hi,

I would like to Beta test when it's time.

When this project advances, I think it should have it's own topic/forum, as this one is '...ASP.NET (Non_Forum Related)...'

I also open a can of worms by asking for core support for binary content in the DB. IMO, at some point, binaries including video, should be in (or linked from) normal postings, not in separate galleries/archives. IMO, storing binaries (with can be enabled / disabled and size limited per upload and per user) is easier to managage than separate file directories if the structre allows simple cleanups, for example for expired accounts, deleted content, archived content, and other dead binaries. Stopping executables, scripts etc, is of course vital.
Go to Top of Page

HuwR
Forum Admin

United Kingdom
20550 Posts

Posted - 13 December 2006 :  10:35:17  Show Profile  Visit HuwR's Homepage
rubbish, if you knew anything about db performance, you would know that storing binaries in a database is an extremely bad idea
Go to Top of Page

pdrg
Support Moderator

United Kingdom
2897 Posts

Posted - 13 December 2006 :  11:27:46  Show Profile  Send pdrg a Yahoo! Message
quote:
Originally posted by janaf


I also open a can of worms by asking for core support for binary content in the DB. IMO, at some point, binaries including video, should be in (or linked from) normal postings, not in separate galleries/archives. IMO, storing binaries (with can be enabled / disabled and size limited per upload and per user) is easier to managage than separate file directories if the structre allows simple cleanups, for example for expired accounts, deleted content, archived content, and other dead binaries. Stopping executables, scripts etc, is of course vital.



Too right this is a can of worms, but with the exception of a configuration management system, I do not see a reason to ever do this. Each binary you save has to spool back to disk to do anything with it anyway so you've got additional steps of streaming it into the field, storing huge BLOBS, streaming it (multiple times?!) back out all on top of the regular overhead of binaries. Think harder about your system design, and you can tidy up files far easier in the filesystem.

The only caveats I mentioned are config management, but also winfs, although it does it invisibly so it's more for completeness than an exception!
Go to Top of Page

janaf
Starting Member

Sweden
19 Posts

Posted - 13 December 2006 :  13:27:48  Show Profile
quote:
Originally posted by HuwR

rubbish, if you knew anything about db performance, you would know that storing binaries in a database is an extremely bad idea



True, I have not seen the perfomance issues, because on the small & medium size systems I have worked with, typiaclly less than a thousand visitors / 10.000 page views per day, performance (due to BLOBs or other) has not been an issue. The BLOBs (50-200K) handle in a blink and I can not see a big percieved performance difference between the Access file saved BLOBs and file system based binary file handling. This may be completely different for large systems.

quote:
Originally posted by kentk...
huge BLOBS, streaming it (multiple times?!) back out all on top of the regular overhead of binaries


We have two issues here, one is if we allow binary uploads or not and the other is how to store it.

As far as I have seen, storage overhead for a BLOB is not large, compared to saving in the FS (wich btw is also a kind of database...) if done right, perhaps 20% space overhead. I just checked with a small DB I have with about 100 BLOB files. The uncompressed Access file size is 32MB, and the sum of the size of BLOB files in it is 28.8MB. I should add 10% for thumbnails so the main BLOB would not nead to be read all the time. Done wrong, the file size can easily increase by 1000% in size when BLOB:ed into the DB.

The disk price cost is now around 1$ per GB and steadily decreasing. By the time a project like this would be ready to run, price wil most likely have gone down a lot more.

For me, and I think many small medium size systems, ease of management both for administrators and site users, is far more important than disk space and performance.

If you think management can be done as well as files on the FS rather than in the DB file, fine with me. The important issue is that the binary file handling is easy to manage and presentation is well integrated, not a loose add-on.

Edited by - janaf on 13 December 2006 13:51:47
Go to Top of Page

HuwR
Forum Admin

United Kingdom
20550 Posts

Posted - 13 December 2006 :  14:07:24  Show Profile  Visit HuwR's Homepage
1) While Filesytems may share some characteristics of a database,it is debatable whether a file system can be classified as a special-purpose database
2) disk cost per Gb is a lot cheaper and far less restricted (in web hosting terms) than database size/cost, and you certainly wouldn't be using an access db for web hosting if it's size was measured in Gb.
3) Performance of binaries stored in a db vs FS is not a disputable issue, it is a well known fact, try doing some research on google, there are hundreds of articles which will tell you storing files in db is a bad idea
4) This is a forum, it is not a file management or a content management system and therefore management of binary files whether they be in the db or in the File system is not an important issue in the slightest.
Go to Top of Page

pdrg
Support Moderator

United Kingdom
2897 Posts

Posted - 13 December 2006 :  14:33:33  Show Profile  Send pdrg a Yahoo! Message
Ahhh - do you mean like storing a photo in Access/Rich client like Northwind does? If you want a web app/thin client/hefty db you have a very different case, I feel. Then you have to manage streaming. Not really worried about disk space, but db bloat can be an issue I'm sure your DBA will have comments about :-D
Go to Top of Page

HuwR
Forum Admin

United Kingdom
20550 Posts

Posted - 13 December 2006 :  14:35:52  Show Profile  Visit HuwR's Homepage
quote:
do you mean like storing a photo in Access/Rich client like Northwind does?
This is still an extremely bad thing to do, especially where web sites are concerned, IIS and all other web servers cache image files for a reason, they do not cache images streamed from a db hence the reason it is a very bad thing to do.
Go to Top of Page

janaf
Starting Member

Sweden
19 Posts

Posted - 13 December 2006 :  16:05:18  Show Profile
Why so offended?

1) I never said it was (but guess the comment was for me).

2) Yes.

3) After searching the web a little, I do not come to the same conclusion. There are plenty of people who LIKE one or the other, but is seems few have actually compared and most references I find talk about the pros and cons of each solution, including terrabyte databases with blobs.

Here is a wiki with summaries by 12 authors:
http://en.wikibooks.org/wiki/Programming:WebObjects/Web_Applications/Development/Database_vs_Filesystem

Here is a june 2006 Microsoft Research paper where they have measured, with SQL Server and the result is a bit "it depends", and the performance difference is not big either way. The Paper also has a chapter named Folklore which starts "There is a wealth of anecdotal experience with applications that use large objects". ftp://ftp.research.microsoft.com/pub/tr/TR-2006-45.pdf

The result summary is: "The study indicates that if objects are larger than one megabyte on average, NTFS has a clear advantage over SQL Server. If the objects are under 256 kilobytes, the database has a clear advantage. Inside this range, it depends on how write intensive the workload is, and the storage age of a typical replica in the system"

Myself, I do not know, I never made a comparison of BLOB performance, and I did not claim I did. I only say that in the limited BLOB use I have done, performance was not a problem or a limiting factor.

4) It seems some of the most visited threads on the mod pages on this site are the file sharing mod thread by korayem, the file sharing mod thread by proeder, the photo album and photo gallery mod threads.
Go to Top of Page

HuwR
Forum Admin

United Kingdom
20550 Posts

Posted - 13 December 2006 :  16:58:53  Show Profile  Visit HuwR's Homepage
1) you did
quote:
compared to saving in the FS (wich btw is also a kind of database...)
sorry, but if you don't even acknowledge what you write then there isn't much point in pursueing it.
2) yes what ?
3) blob performance may not be a limiting factor in certain circumstances, but when it comes to web servers it is since as I said IIS and other web servers cache static files and images, streamed blob fields are not static by there nature are not cached, this is extremely limiting when you are paying for your bandwidth.
4) So what,because a thread is the most visited does not necesarily imply that it is the most popular Mod, it could mean that it is either the most buggy or the most difficult to implement, so you should not read anything into the viewing numbers of a topic, and besides just because people utilise those mods does not make it the business of a forum to be a file management system, you may well find that most people who utilise these mods do so as part of a portal system, Snitz is not a portal system it is a forum there is a distinction.
Go to Top of Page

janaf
Starting Member

Sweden
19 Posts

Posted - 13 December 2006 :  18:39:55  Show Profile
quote:
Originally posted by HuwR

1) While Filesytems may share some characteristics of a database,it is debatable whether a file system can be classified as a special-purpose database


I did not write it was could be "classified as a special-purpose database", but "wich btw is also a kind of database..." which is something else.
quote:
Originally posted by HuwR

2) disk cost per Gb is a lot cheaper and far less restricted (in web hosting terms) than database size/cost, and you certainly wouldn't be using an access db for web hosting if it's size was measured in Gb.

2) yes, I agree to your 2) statement
quote:
Originally posted by HuwR
3) blob performance may not be a limiting factor in certain circumstances, but when it comes to web servers it is since as I said IIS and other web servers cache static files and images, streamed blob fields are not static by there nature are not cached, this is extremely limiting when you are paying for your bandwidth.
4) So what,because a thread is the most visited does not necesarily imply that it is the most popular Mod, it could mean that it is either the most buggy or the most difficult to implement, so you should not read anything into the viewing numbers of a topic, and besides just because people utilise those mods does not make it the business of a forum to be a file management system, you may well find that most people who utilise these mods do so as part of a portal system, Snitz is not a portal system it is a forum there is a distinction.


3) I can not see how caching in the web server changes the bandwidth use, unless it is browser caching? But then I do not know much of how IIS works. But both the OS and SQL server and even the disk also cache, so I would like to see some real life tests before drawing conclusions. I just may do some tests one day. To some, bandwidth cost is limitig, and you would always have the possibility to shut binary files off or on, regardless of how they are stored. And like I wrote before, if this can be solved better by storing in the FS, fine with me.

4) Could be, I don't know. It could also be that they simply are the most popular. One indication is the mod downloads, where at Snitzbitz the Avatar Mod is topping the downloads list right now.

A .net port should i.m.o. aim for some guestimate of what the situation (performance, demands, costs), will be in something like 5 years from now, not how it is now. What has been the last years, and how it is right now will be passed tense when a .net port is available. Look back a few years and extrapolate forward a few years from now.

My guess, based on my own needs and requests I see coming from customers and users, is that a more or less text focused forum system will not be very interesting in five years, whether we like it or not.

And again there are two separate issues, one if binary content should be intergarted/promoted/accepted, the other is how to store it.

But I as I can not spend time on development right now, I will have to sit back and see what comes, and try to like it !

My two cents worth....
Go to Top of Page
Page: of 11 Previous Topic Topic Next Topic  
Previous Page | Next Page
 New Topic  Topic Locked
 Printer Friendly
Jump To:
Snitz Forums 2000 © 2000-2019 Snitz™ Communications Go To Top Of Page
This page was generated in 0.09 seconds. Powered By: Snitz Forums 2000 Version 3.4.07