Author |
Topic  |
Jeepaholic
Average Member
  
USA
697 Posts |
Posted - 17 September 2003 : 05:44:18
|
04/29/04 UPDATE: Realized that the Events Calendar mod database fields are included in this release. If you do not have this mod, you need to eliminate all references to T_EVENT_DATE and T_ISEVENT.
As well, if you have other mods that only add columns to the TOPIC and/or REPLY tables, you could just as easily add them to this script in order to assure that information is archived as well.
11/22/03 UPDATE: It just dawned on me that the script does not lock the topics as they are archived. This is a requirement for the system. Please make the change (in red) to the script below.
11/22/03 UPDATE: An UN-Archive script has been created as well, and is located in THIS TOPIC
11/19/03 UPDATE: The code appears to be functional and accurate. It is posted below in green. You can use Query Analyzer to run this query on your forums, and it will archive and update the forum counts VERY quickly compared to the default forum archiving functionality. My forums would actually time-out and fail trying to archive a single month, however, with this query it took around 30 seconds to archive nearly 6 months. Your mileage will, of course, vary based on hardware. Simply enter the date in the format shown (to archive from), and the Forum ID to archive.
As stated, I believe it's accurate in what it does...but as always...backup your database first, test it on a copy of your database if you can, and use at your own risk.
ORIGINAL POST Hey there. I've archived my forum a number of times - but for some reason, I can't seem to archive anything. I click "Yes" that I'm sure I want to archive the selected topics, and the browser just spins and spins. MS SQL Server doesn't seem to be showing any form of a "hit" regarding the archiving process.
I'll try to provide some stats, but I guess I'm looking for a way to debug this. My site performance is taking a hit, and am in desperate need of archiving again.
STATS: 224,407 Posts in 20,610 Topics 156,966 Archived Posts in 13,517 Archived Topics
I have tried to archive a forum with only 100 posts, which seems to have worked...HOWEVER, the archive topics admin screen never updated to tell me it was complete. Still spinning, in fact.
In my larger forums, I've also tried to only archive one month at a time, even though I ultimately need to archive 3-4 months worth. No avail, same issue.
Any help or words of wisdom would be appreciated. Thanks! Al
CODE
DECLARE
@ArchiveDate nvarchar(14),
@ArchiveForumID int
SET
@ArchiveDate = '20030701000000' -- Date to archive from (YYYYMMDDHHMMSS)
SET
@ArchiveForumID = 35 -- Forum ID to Archive
-- Create temp table that will hold TOPIC_ID's to archive
CREATE TABLE #TOPIC_IDs
(
TOPIC_ID int
)
-- Fill temp table with TOPIC_ID's to archive
INSERT INTO
#TOPIC_IDs
-- Query finds topics started (and edited) before archive date
SELECT
TOPIC_ID
FROM
FORUM_TOPICS AS FT
WHERE
T_DATE < @ArchiveDate
AND
-- If no last edit date (NULL), return lowest date possible
ISNULL(T_LAST_EDIT,'00000000000000') < @ArchiveDate
AND
FORUM_ID = @ArchiveForumID
AND
-- Subquery to determine if any replies (or edits) are
-- AFTER the archive date. If so, do not archive topic.
(SELECT
COUNT(*)
FROM
FORUM_REPLY AS FR
WHERE
(R_DATE >= @ArchiveDate
OR
-- If no last edit date (NULL), return lowest date possible
ISNULL(R_LAST_EDIT,'00000000000000') >= @ArchiveDate)
AND
FR.TOPIC_ID = FT.TOPIC_ID
) = 0
-- Copy TOPIC's to archive table
INSERT INTO
FORUM_A_TOPICS
(CAT_ID,
FORUM_ID,
TOPIC_ID,
T_STATUS,
T_MAIL,
T_SUBJECT,
T_MESSAGE,
T_AUTHOR,
T_REPLIES,
T_UREPLIES,
T_VIEW_COUNT,
T_LAST_POST,
T_DATE,
T_LAST_POSTER,
T_IP,
T_LAST_POST_AUTHOR,
T_LAST_POST_REPLY_ID,
T_ARCHIVE_FLAG,
T_LAST_EDIT,
T_LAST_EDITBY,
T_STICKY,
T_SIG,
T_EVENT_DATE,
T_ISEVENT)
SELECT
CAT_ID,
FORUM_ID,
TOPIC_ID,
0,
T_MAIL,
T_SUBJECT,
T_MESSAGE,
T_AUTHOR,
T_REPLIES,
T_UREPLIES,
T_VIEW_COUNT,
T_LAST_POST,
T_DATE,
T_LAST_POSTER,
T_IP,
T_LAST_POST_AUTHOR,
T_LAST_POST_REPLY_ID,
T_ARCHIVE_FLAG,
T_LAST_EDIT,
T_LAST_EDITBY,
T_STICKY,
T_SIG,
T_EVENT_DATE,
T_ISEVENT
FROM
FORUM_TOPICS
WHERE
TOPIC_ID
IN
(SELECT
TOPIC_ID
FROM
#TOPIC_IDs)
-- Copy REPLY's to archive table
INSERT INTO
FORUM_A_REPLY
(CAT_ID,
FORUM_ID,
TOPIC_ID,
REPLY_ID,
R_MAIL,
R_AUTHOR,
R_MESSAGE,
R_DATE,
R_IP,
R_STATUS,
R_LAST_EDIT,
R_LAST_EDITBY,
R_SIG)
SELECT
CAT_ID,
FORUM_ID,
TOPIC_ID,
REPLY_ID,
R_MAIL,
R_AUTHOR,
R_MESSAGE,
R_DATE,
R_IP,
R_STATUS,
R_LAST_EDIT,
R_LAST_EDITBY,
R_SIG
FROM
FORUM_REPLY
WHERE
TOPIC_ID
IN
(SELECT
TOPIC_ID
FROM
#TOPIC_IDs)
-- Delete TOPIC's from main table
DELETE
FROM
FORUM_TOPICS
WHERE
TOPIC_ID
IN
(SELECT
TOPIC_ID
FROM
#TOPIC_IDs)
-- Delete REPLY's from main table
DELETE
FROM
FORUM_REPLY
WHERE
TOPIC_ID
IN
(SELECT
TOPIC_ID
FROM
#TOPIC_IDs)
DROP TABLE #TOPIC_IDs
-- Update FORUM_TOTALS Counts
UPDATE
FORUM_TOTALS
SET
P_COUNT = (
SELECT
COUNT(*)
FROM
FORUM_REPLY
),
P_A_COUNT = (
SELECT
COUNT(*)
FROM
FORUM_A_REPLY
),
T_COUNT = (
SELECT
COUNT(*)
FROM
FORUM_TOPICS
),
T_A_COUNT = (
SELECT
COUNT(*)
FROM
FORUM_A_TOPICS
)
-- Update Specific Forum Counts
UPDATE
FORUM_FORUM
SET
F_TOPICS = (
SELECT
COUNT(*)
FROM
FORUM_TOPICS
WHERE
FORUM_ID = @ArchiveForumID
),
F_COUNT = (
SELECT
COUNT(*)
FROM
FORUM_REPLY
WHERE
FORUM_ID = @ArchiveForumID
),
F_A_TOPICS = (
SELECT
COUNT(*)
FROM
FORUM_A_TOPICS
WHERE
FORUM_ID = @ArchiveForumID
),
F_A_COUNT = (
SELECT
COUNT(*)
FROM
FORUM_A_REPLY
WHERE
FORUM_ID = @ArchiveForumID
)
WHERE
FORUM_ID = @ArchiveForumID
GO
|
Al Bsharah Aholics.com
Jeepaholics Anonymous Broncoholics Anonymous Network Insight
|
Edited by - Jeepaholic on 29 April 2004 21:55:36 |
|
Jeepaholic
Average Member
  
USA
697 Posts |
|
Nikkol
Forum Moderator
    
USA
6907 Posts |
|
Jeepaholic
Average Member
  
USA
697 Posts |
|
ruirib
Snitz Forums Admin
    
Portugal
26364 Posts |
Posted - 17 September 2003 : 17:32:33
|
I know this does not help much, but I wouldn't be surprised if this was due to the number of records in the forum. Al, maybe try selecting the shortest amount of time possible and choose the smaller forum, just to see if it works. |
Snitz 3.4 Readme | Like the support? Support Snitz too |
 |
|
Jeepaholic
Average Member
  
USA
697 Posts |
Posted - 17 September 2003 : 17:38:41
|
Hi Rui!
From my first post:
quote: I have tried to archive a forum with only 100 posts, which seems to have worked...HOWEVER, the archive topics admin screen never updated to tell me it was complete. Still spinning, in fact.
In my larger forums, I've also tried to only archive one month at a time, even though I ultimately need to archive 3-4 months worth. No avail, same issue.
I have tried both of those items, with the results shown above. Any thoughts on how to resolve/debug this? If it's due to the number of records in the forum (which seems very likely), how can I go about archiving? I've limited the number of records to archive as much as I possibly can, and it's still not working... |
Al Bsharah Aholics.com
Jeepaholics Anonymous Broncoholics Anonymous Network Insight
|
 |
|
ruirib
Snitz Forums Admin
    
Portugal
26364 Posts |
|
Jeepaholic
Average Member
  
USA
697 Posts |
Posted - 17 September 2003 : 22:25:51
|
Thanks Rui. Any chance using the Profiler would give us more info? I'm not very familiar with how to use it, but with some guidance I could set something up and see what comes of it. Let me know if there's anything else I can provide...thanks! |
Al Bsharah Aholics.com
Jeepaholics Anonymous Broncoholics Anonymous Network Insight
|
 |
|
ruirib
Snitz Forums Admin
    
Portugal
26364 Posts |
Posted - 18 September 2003 : 09:47:15
|
Well clearly we need to findout what part of the script is being run, what part is getting stuck. With the profiler we may able to find that. So, if you want to run a trace on that, it could help. Otherwise we'll have to resort to old-fashioned debugging, which can make it hard for me to help you, since I don't have a DB with that much data.
Anyway I think I've found room for improvement in the script, but it does not make any sense to do it if the parts in question are not causing the current problems. Let's identify the causes and act with that info on hand. |
Snitz 3.4 Readme | Like the support? Support Snitz too |
 |
|
Jeepaholic
Average Member
  
USA
697 Posts |
Posted - 18 September 2003 : 13:51:41
|
Ok, I'm going to need some help setting up the Profiler. What should I filter on to make sure I only get Archiving Information? I've never really set up anything other than a basic "watch everything" with the profiler.
Thanks Rui!
Completely side note, but for my own curiosity, how does the script work - in general? It appears that it doesn't just copy each line from the main table to the archive table? What are the steps it takes? |
Al Bsharah Aholics.com
Jeepaholics Anonymous Broncoholics Anonymous Network Insight
|
 |
|
ruirib
Snitz Forums Admin
    
Portugal
26364 Posts |
Posted - 18 September 2003 : 17:38:40
|
SQL:BatchCompleted events are enough for me to find out where it gets stuck.
The script basically gets the replies, navigates the recordset in order to insert them one by one and writes each one into the archived replies table. Then navigates the recordset again to delete the replies from the live replies table(improvement can be achieved here). Then gets the topics and inserts them in the archived topics table one by one. After that deletes the topics from the live table. It also updates counts after doing this. |
Snitz 3.4 Readme | Like the support? Support Snitz too |
 |
|
Jeepaholic
Average Member
  
USA
697 Posts |
Posted - 18 September 2003 : 23:25:42
|
Ok...started running one and it's up to 2000 lines after only about 5 minutes. Shall I continue with it, or is there a way to filter futher?
I can also try late at night when there's less traffic, if there's no better way to filter. |
Al Bsharah Aholics.com
Jeepaholics Anonymous Broncoholics Anonymous Network Insight
|
Edited by - Jeepaholic on 18 September 2003 23:27:25 |
 |
|
Jeepaholic
Average Member
  
USA
697 Posts |
Posted - 19 September 2003 : 01:49:22
|
Ok Rui, I've got like 25000 lines in a 9MB trace file on a failed archive. If you want that shoot me an email and I'll give you FTP access. Or, if there's a better way to filter let me know and we'll start over. Thanks again for any help... |
Al Bsharah Aholics.com
Jeepaholics Anonymous Broncoholics Anonymous Network Insight
|
 |
|
ruirib
Snitz Forums Admin
    
Portugal
26364 Posts |
Posted - 19 September 2003 : 08:01:51
|
Al, is that a 9 MB trace just for the archive function? Or does it include the usual forum traffic?
It would help if you could choose a period of less traffic at the forum and then run the archive procedure and stop the trace after a couple minutes. Of course, I'm mainly interested in seeing how the SQL statements related to the archiving script are being handled, to see where the script is hanging. |
Snitz 3.4 Readme | Like the support? Support Snitz too |
 |
|
Jeepaholic
Average Member
  
USA
697 Posts |
|
Jeepaholic
Average Member
  
USA
697 Posts |
Posted - 20 September 2003 : 03:01:44
|
Shoot me an email, Rui. I'll reply with the new file. Ran for about 30 minutes...2700 lines (1.7MB I think). It was an attempt to archive less than 1 months worth of data... If that's too big, let me know and I'll try something else. |
Al Bsharah Aholics.com
Jeepaholics Anonymous Broncoholics Anonymous Network Insight
|
 |
|
Topic  |
|
|
|