Sorry your browser is not supported!

You are using an outdated browser that does not support modern web technologies, in order to use this site please update to a new browser.

Browsers supported include Chrome, FireFox, Safari, Opera, Internet Explorer 10+ or Microsoft Edge.

Geek Culture / PHP - writing a forum - maybe Rich can answer this?

Author
Message
David T
Retired Moderator
22
Years of Service
User Offline
Joined: 27th Aug 2002
Location: England
Posted: 19th Apr 2004 21:30
Hi,

I'm currently having a go at updating some old forum code I wrote last year. And I'm wondering, how does Apollo / every other forum know which topics the member has read and which he hasn't?

The only way I can think of is storing a record every time the user views a post, then querying for every topic to find out if you've replied to it and whether there are any new posts. But, that would mean an obscene amount of queries per page.

Maybe Rich can shed some light on how he did it?

Thanks in advance

"To do is to be" - Descartes
"To be is to do" - Voltaire
"Do be do be do" - Frank Sinatra
Rob K
Retired Moderator
22
Years of Service
User Offline
Joined: 10th Sep 2002
Location: Surrey, United Kingdom
Posted: 19th Apr 2004 21:45 Edited at: 19th Apr 2004 21:56
The way I'd do it would be to store the number of posts there were in the thread the last time the user viewed it (including the first post)

When a new thread is created, the thread has 1 post, but the user's "posts last time I saw it" value will be 0 - therefore the thread has been updated.

When you click to view the post, that count goes up to however many posts there are.

Let's say there are 20 threads per page, you obviously only need to check the threads for that page.

Then you go through the threads and select entries from the "posts last time I saw it" table with the same IDs.

This means just a couple of queries in all *I think*. I'm assuming that you can use AND / OR operators with the WHERE clause but I've not had to do it myself.

BTW. I'm assuming that any "deleted" posts are not really deleted, but just made invisible.

BlueGUI:Windows UI Plugin - All the power of the windows interface in your DBPro games. - Plus URL download, win dialogs.
Over 140 new commands
David T
Retired Moderator
22
Years of Service
User Offline
Joined: 27th Aug 2002
Location: England
Posted: 19th Apr 2004 22:07 Edited at: 19th Apr 2004 22:09
Hmm. Kind of along the same lines that I thought.

THe problem woud be having to store for each thread the last known number of posts for every user. If I had 25 members with 200 threads, that's 5000 additional table rows, unless some cleverness with SQL can be done.

One thought is, have a table that records number of posts, and in it have one row for every topic. Each row contains one field for every user, with new fields being added as new users join.

For example

user 1 user 2
TOPIC 1 | 2 3
TOPIC 2 | 0 0

(user 3 now joins)

user 1 user 2 user 3
TOPIC 1 | 2 4 0
TOPIC 2 | 0 0 0

As long as I downloaded all the records at the beginning then sifted through them using PHP, it's easy to get numbers and adds only one extra query.

"To do is to be" - Descartes
"To be is to do" - Voltaire
"Do be do be do" - Frank Sinatra
Richard Davey
Retired Moderator
22
Years of Service
User Offline
Joined: 30th Apr 2002
Location: On the Jupiter Probe
Posted: 19th Apr 2004 22:32
David - the way Apollo does it is as follows:

I have a user_board_view_history table and a user_thread_view_history table. They are small, Fixed Length MySQL tables that consist of very little by references:

For the Board one:

user_ref - int(4)
forum_ref - tinyint
board_ref - tinyint
marked_read - datetime

For the Thread one:

user_ref - int(4)
thread_ref - int(4)
board_ref - tinyint
last_read - datetime
last_view - datetime

When a user views a thread they have never seen before it will add an entry into the thread_history table recording the last_read date and the last_view date. last_view is the actual time the user viewed the thread, last_read is the timestamp of the LAST POST the user read in that thread.

In order to curb a massive table building up, when-ever someone marks a BOARD as being read - I will run a DELETE query on the Thread History, removing any views of threads in that given board prior to the current date. This automatically keeps your thread_history table clean.

Even if no user ever did that, you're still only talking about a very tiny table size anyway. My thread_history table for example will only take up 14KB of database size per 300 records. So even at 5000 records that's only 200KB - and remember, it is a FIXED Length table, so MySQL will index and seek on that extremely quickly. You could also have a cron job scheduled to automatically purge this table every 30 days if you wanted (i.e. auto mark-as-read threads older than say 90 days).

Because I have an index on the user_ref combined with the thread_ref, I can do a SELECT .. IN () query on the table, passing in the ID of all threads displayed on the current page and it'll bring back the results in mere miliseconds. It need only be 1 query in total to get this "last viewed" information.

The reason I have a last_read and a last_viewed is that the last_read date stores the actual date of the most recent message read in that thread - whereas last_viewed stores the last time the user actually viewed the thread. If you don't do it like this, you will mark a whole thread as "being read" if they only look at the very first page of it, when in actual fact the "un-read" part of it might be on a say page 3 or 4.

If you want any more info, shout

Cheers,

Rich

With our species on the edge of extermination,
with no prospect but a horrible death,
we actually played games.
BatVink
Moderator
21
Years of Service
User Offline
Joined: 4th Apr 2003
Location: Gods own County, UK
Posted: 19th Apr 2004 22:43 Edited at: 19th Apr 2004 22:44
[Edit] I'll leave this here, but Rich hadn't replied when I started!

David, I don't think your last idea is the right way to do it. A database structure should be static 99% of the time, only the data should be fluid.

Rob K's idea sounds OK. You need a table with only 3 fields. Additionally, you only need a record when a user views a thread. No record means never viewed. This should save a great deal of space.

Remember, 90% of registrations will die very quickly, so you don't want the overhead of recording unneccesary data for dead users.

The only concern I have is this: I don't know how MySQL deals with not finding a record. As a general rule of thumb, you should try to write queries are highly likely to find a match, as this is more efficient than not finding a match.

BatVink
http://facepaint.me.uk/catalog/default.php
AMD 3000+ Barton, 512K Ram, 120 Gig Drive space, GeForce 5200 FX 128 Mb, Asus A7N8X Mobo.
David T
Retired Moderator
22
Years of Service
User Offline
Joined: 27th Aug 2002
Location: England
Posted: 19th Apr 2004 23:59 Edited at: 20th Apr 2004 00:15
Quote: "David, I don't think your last idea is the right way to do it. A database structure should be static 99% of the time, only the data should be fluid."


I too thought it seems a little ropey

Thanks for the reply Rich, I'll look into SELECT ... IN()

[edit]

Okay, I've researched the IN statrement - you wouldn't believe how much trouble I had finding it on Google

"To do is to be" - Descartes
"To be is to do" - Voltaire
"Do be do be do" - Frank Sinatra
adr
21
Years of Service
User Offline
Joined: 21st May 2003
Location: Job Centre
Posted: 20th Apr 2004 12:46
Rich : In your experience, do tinyints actually make a speed difference? I can appreciate that they make an impact on storage, but I'm interested in optimizing our DB for speed. Our database system is rather large (one db is over 10GB) and while I can't do much about the size, I'd sure as hell like to speed it up

stop();
hammertime();
David T
Retired Moderator
22
Years of Service
User Offline
Joined: 27th Aug 2002
Location: England
Posted: 20th Apr 2004 14:54
Okay, I've come up with this.

My tables:

pb_topic_views
- topicid
- userid
- lastview // timestamp, last time we viewed topic

pb_topics
(among others...)
- topicid
- lastmodified // timestamp, last time the topic was replied to

And this is the query:

SELECT t.topicid, t.lastmodified, v.lastview
FROM pb_topic_views v, pb_topics t
WHERE t.topicid = v.topicid AND v.userid = {viewer's user id}
ORDER BY t.lastmodified DESC
LIMIT x,y

Which would (hopefully) return the topic id, last view and last update dates for the topic.

The ORDER BY and LIMIT statements order the results, and limit the results returned to the number that we got when querying the topic table.

Come to think of it, could I change

SELECT t.topicid, t.lastmodified, v.lastview
to

SELECT t, v.lastview

So I get all topic info returned as normal, plus the date we last viewed each topic?

"To do is to be" - Descartes
"To be is to do" - Voltaire
"Do be do be do" - Frank Sinatra
David T
Retired Moderator
22
Years of Service
User Offline
Joined: 27th Aug 2002
Location: England
Posted: 20th Apr 2004 16:12 Edited at: 20th Apr 2004 16:14
Okay just to add one more question

I have the following query:

$sql = "SELECT * FROM pb_topics t, pb_topic_views v WHERE v.id = t.id AND t.board = '".$boardid."' ORDER BY t.lastmodified DESC ".$limittx;

($limittx is just a string containing the calculated LIMIT statement)

And it returns an array of all the topic plus the date on which they were viewed (I haven't included checking which user viewed them yet).

The problem is this does not return topics where viewed record exists for the topic (i.e we've never viewed it before) and so then upsets all the code below it.

Is there a way to join the table if a matching row does exist, otherwise just continue as normal giving us the unjoined ones?

Oh, and this is my basic code I use to view fourms / topic etc.



But it returns an error when the array is empty. Is there a way to silence this error without writing extra code to check to see if the array is empty?

"To do is to be" - Descartes
"To be is to do" - Voltaire
"Do be do be do" - Frank Sinatra
Karlos
22
Years of Service
User Offline
Joined: 18th Nov 2002
Location: United Kingdom
Posted: 20th Apr 2004 16:22
i would personally wrap the category display in an if ($cats) {

or php equivalent. only 1 more command

If it ain't broke - try harder.
XP Pro - Radeon 9000 Mobility- P4 3.0ish
Football management - Football Manager
David T
Retired Moderator
22
Years of Service
User Offline
Joined: 27th Aug 2002
Location: England
Posted: 20th Apr 2004 17:23
Okay, thanks.

"To do is to be" - Descartes
"To be is to do" - Voltaire
"Do be do be do" - Frank Sinatra
Richard Davey
Retired Moderator
22
Years of Service
User Offline
Joined: 30th Apr 2002
Location: On the Jupiter Probe
Posted: 20th Apr 2004 17:39
adr - "do tinyints actually make a speed difference?"

Sure, as well as taking less disk space MySQL also knows it can retrieve the whole value in a single CPU tick because the length of the values fits into 32-bits (CMP EAX, EBX).

David - the way you've done it will work but only if a record exists for that thread in the pb_topic_views table. If you do a LEFT JOIN instead you will get back all the threads, regardless if they have been viewed or not.

Also - recording just the lastview date for a thread will break when that thread spans multiple pages. I.e. on a 6 page thread, if they view the first page (which doesn't have anything un-read on it) then it will log the last view date which will automatically mark the rest of the thread as being read, even though the user has never seen the new posts.

Make sense?

Also: $sql = "SELECT * FROM pb_topics t, pb_topic_views v WHERE v.id = t.id AND t.board = '".$boardid."' ORDER BY t.lastmodified DESC ".$limittx;

You don't need to jump in and out of the string so much. Rather:

$sql = "SELECT * FROM pb_topics t, pb_topic_views v WHERE v.id = t.id AND t.board = '$boardid' ORDER BY t.lastmodified DESC $limittx";

You can wrap your $cats in an if block if you like, but it'll only work if you return a boolean value of false on error.

Cheers,

Rich

With our species on the edge of extermination,
with no prospect but a horrible death,
we actually played games.
David T
Retired Moderator
22
Years of Service
User Offline
Joined: 27th Aug 2002
Location: England
Posted: 20th Apr 2004 18:19 Edited at: 20th Apr 2004 18:22
Hi,

Thanks - The LEFT JOIN worked like a charm

I know I neednt jump in and out, just makes me feel better

You can see the forum here:
http://davidtattersall.me.uk/pb2/

I admit I took a little inspiration from Apollo, I just like it so much
And I did nick the colour code for that blue colour.
And the little metal bar at the top is just a placeholder until I get my own.

If you browse the Powerboard Talk forum, all topics should appear as unread. Login with the user id 1 and you should see thge topics in that forum change to read, unread and updated

Finally, I probably will eventually only store a "hit" on the topic when the last post is viewed, meaning that missing the last page will still mean that it comes up as unread.

However, following individual hits on posts will be coming later, and with that will come Apollo style colouring of posts to show which ones have been viewed.

"To do is to be" - Descartes
"To be is to do" - Voltaire
"Do be do be do" - Frank Sinatra
Richard Davey
Retired Moderator
22
Years of Service
User Offline
Joined: 30th Apr 2002
Location: On the Jupiter Probe
Posted: 20th Apr 2004 18:28
Looking nice

And hey.. copying is flattery 'n all that!

Post again if you need more help - makes such a change from "build me a MMORPG"

Cheers,

Rich

With our species on the edge of extermination,
with no prospect but a horrible death,
we actually played games.
David T
Retired Moderator
22
Years of Service
User Offline
Joined: 27th Aug 2002
Location: England
Posted: 20th Apr 2004 18:30 Edited at: 20th Apr 2004 18:33
By the way, I was wondering, can you make Halo 2 in "BDPro"?



[edit]

Seriously now, can you LEFT JOIN with more than two tables?

I'm thinking of bundling all the information about the person who posted the topic as well as the first post of the topic in with the results returned from above.

"To do is to be" - Descartes
"To be is to do" - Voltaire
"Do be do be do" - Frank Sinatra
Richard Davey
Retired Moderator
22
Years of Service
User Offline
Joined: 30th Apr 2002
Location: On the Jupiter Probe
Posted: 20th Apr 2004 20:51 Edited at: 20th Apr 2004 20:52
You can make HALO2 in PHP didn'tcha know?

Yes you can LEFT JOIN more than once, here is an example query from Apollo:

SELECT
*
FROM
user_profile
LEFT JOIN user_profile_extended ON user_profile_extended.user_ref = user_id
LEFT JOIN user_preferences ON user_preferences.user_ref = user_id
WHERE
site_user_id = 'a9dab5cc9dbb2dd216baae1a9ec25683'

Cheers,

Rich

With our species on the edge of extermination,
with no prospect but a horrible death,
we actually played games.
David T
Retired Moderator
22
Years of Service
User Offline
Joined: 27th Aug 2002
Location: England
Posted: 20th Apr 2004 21:05
Halo 2 in PHP?
Gimme teh codez!

Thanks

"To do is to be" - Descartes
"To be is to do" - Voltaire
"Do be do be do" - Frank Sinatra
David T
Retired Moderator
22
Years of Service
User Offline
Joined: 27th Aug 2002
Location: England
Posted: 24th Apr 2004 00:05
Back again!


Rich, I'm just wondering about the topic views system you have.

You say to keep database sizes down you offer a "mark as read" button that removes references from the views field - but wouldn't that set topics as viewed again?

Thanks

"To do is to be" - Descartes
"To be is to do" - Voltaire
"Do be do be do" - Frank Sinatra
Richard Davey
Retired Moderator
22
Years of Service
User Offline
Joined: 30th Apr 2002
Location: On the Jupiter Probe
Posted: 25th Apr 2004 15:22
Nope because I log the "last board marked read" date as well as the "last viewed date". So when someone lists the threads in a board I do this:

1) Get the last time they marked this board as read (if ever) (A)
2) Get the last viewed date for the threads (B) (if ever)
3) If A > B then thread is old, otherwise thread is new

Cheers,

Rich

With our species on the edge of extermination,
with no prospect but a horrible death,
we actually played games.
David T
Retired Moderator
22
Years of Service
User Offline
Joined: 27th Aug 2002
Location: England
Posted: 25th Apr 2004 18:15 Edited at: 25th Apr 2004 18:17
Okay, cheers.

One last thing:

Is there any way to change the value of a text field via javascript? The text field is on a page, then clicking a link opens a popup which allows you to choose an option. Choosing the option then modifies the text box value on the original parent page.

I think it's possible, I'll dig out one of my books

"To do is to be" - Descartes
"To be is to do" - Voltaire
"Do be do be do" - Frank Sinatra

Login to post a reply

Server time is: 2024-11-25 03:41:40
Your offset time is: 2024-11-25 03:41:40