ListPages is very slow to process

Bugs » #16


This section on the Community is no longer supported, in favour of Wikidot's Official Feedback Site .

It is retained here for archiving purposes.

Posted by cold_blood3d cold_blood3d on 21 Jun 2008 23:23, last edited on 04 Mar 2009 09:41

This bug has been fixed

Not completely fixed. Still happens some of the times.

Description

The ListPages module, under certain conditions, is extremely slow to load/process (sometimes taking a couple of minutes). This appears to be a processing lag on Wikidot's end rather than a connection speed issue. The lag occurs even on page lists that consist only of text and would normally take less than a second to load. The time for a list pages to process appears to increase dramatically when ListPages searches a category with a lot of pages.

Once a page with ListPages loads, its contents appear to stay "cached" by Wikidot. Therefore, it only affects the first visitor to the page, as well as visitors that go to the page after it leaves the Wikidot "cache".

Here is a demonstration of what happens:

  • Person A visits a ListPages page, waits 1-2 minutes for it to process and load, then leaves the site.
  • About 15 minutes later, person B goes to the same page. Person B will see the page instantly.

As seen in this example, the initial visit triggers Wikidot to process the ListPages and keep it cached. All visitors coming soon after person A will be able to view the page instantly. I have tested this using multiple computers to prove that it is not a matter of data being cached locally.

This is a problem because:

  • The first person will likely not wait 2 minutes and will quickly leave, so the page will never get cached by Wikidot.
  • The pages don't stay cached forever… If person B (in above example) went back to the page 4 hours later, they would have to endure the massive wait while Wikidot reprocesses it (assuming no one else visited it right before them).

How to Reproduce

To expose this problem, it's best if you have a lot of pages. Here's an example from my wiki, which currently has 200+ pages in the category ListPages is searching.

As described above, once a page is visited it will load fast for all subsequent visitors (only for a while). I included several copies of my page so you can find one that loads slowly.

http://scmapdb.wikidot.com/tag:boms-listpagetest
http://scmapdb.wikidot.com/tag:boms-listpagetest2
http://scmapdb.wikidot.com/tag:boms-listpagetest3
http://scmapdb.wikidot.com/tag:boms-listpagetest4
http://scmapdb.wikidot.com/tag:boms-listpagetest5
http://scmapdb.wikidot.com/tag:boms-listpagetest6
http://scmapdb.wikidot.com/tag:boms-listpagetest7
http://scmapdb.wikidot.com/tag:boms-listpagetest8

See this thread in which other users confirm the bug.

Browsers

Experienced in all browsers.

Workarounds

To keep your ListPages loading fast for everyone, you can:

  • Manually visit the page constantly throughout the day, so Wikidot keeps caching it.
  • Setup some kind of program or bot to automatically visit your pages that have a ListPages module.

Note: The wait time is drastically reduced if you paginate your content by using a low "perPage" value.

Contact

mmclean89 does not match any existing user name

Rate this Bug

Rate the urgency of this bug. If you think it is more urgent and important than it's current rating suggests, rate it up.

rating: +15
probably related to tags
_alf_ _alf_ 05 Jul 2008 00:12

I suffer from this issue too if I specify both category and tag in ListPages.
For example, the following returns a reasonably prompt output

[[module ListPages category="*"]]

whereas the following results in a latency as described in this bug report
[[module ListPages category="*" tag="some_tag"]]
by _alf_ _alf_ , 05 Jul 2008 00:12
Re: probably related to tags

You might be on to something. I've never seen my main page lag (has no tag limts), but my secondary blog does have tag limits, and it does hang on me sometimes. I used to think it was just the accompanying RSS feeds, but I recently separated them and the blog can still hang.

by David Marseilles David Marseilles , 05 Jul 2008 19:27
Gabrys Gabrys 06 Jul 2008 11:14

Hi, we'll take a look at this soon.


Piotr Gabryjeluk
visit my blog

by Gabrys Gabrys , 06 Jul 2008 11:14
Re: ListPages is very slow to process

Thank you =)

by David Marseilles David Marseilles , 06 Jul 2008 18:54
FIXED
Gabrys Gabrys 07 Jul 2008 08:04

Hi all,

we've fixed this issue.

Please verify this.


Piotr Gabryjeluk
visit my blog

by Gabrys Gabrys , 07 Jul 2008 08:04
Re: FIXED

Off-hand, my tag limited page did just load with no lag.

Was the problem related to tags?

by David Marseilles David Marseilles , 07 Jul 2008 08:13
Re: FIXED
cold_blood3d cold_blood3d 07 Jul 2008 19:39

All my pages with ListPages now load pretty much instantly. Thanks so much!

by cold_blood3d cold_blood3d , 07 Jul 2008 19:39
Re: FIXED
_alf_ _alf_ 08 Jul 2008 19:24

I confirm it.
I don't observe the latency I experienced before.

by _alf_ _alf_ , 08 Jul 2008 19:24
Re: FIXED
Sulfrum Sulfrum 13 Jul 2008 07:10

Yes, it works!

Ent


You'll pay for all your sins. If you already paid, please disregard this message.

by Sulfrum Sulfrum , 13 Jul 2008 07:10
Re: FIXED
Sulfrum Sulfrum 13 Jul 2008 19:24

Today when I was using the module I found a bug (can someone confirm that it's really a bug?).
At the bottom of the message there is the link and the source of the page.

Description of the bug:
I use the ListPagesModule with collapsible formatting. If there are some other collapsible before the module, when I click on the collapsible in the module it doesn't open; It open the first or the second. There are no way to open the first and second collapsible in the module.

Sulfrum

This is the source of this page:

+ Collapsible 1
[[collapsible show="+ open" hide="- close"]]
UNO
[[/collapsible]]
+ Collapsible 2
[[collapsible show="+ open" hide="- close"]]
DUE
[[/collapsible]]
+ ListPagesModule with collapsible formatting
[[module ListPages category="obj" tags="ar_accademia-magia -old_obj" order="titleAsc" rssShow="no" perPage="10" separate="yes"]]
[[collapsible show="+ apri" hide="- nascondi"]]
%%content%%
[[/collapsible]]
[[/module]]

You'll pay for all your sins. If you already paid, please disregard this message.

by Sulfrum Sulfrum , 13 Jul 2008 19:24
Re: FIXED
+ ListPagesModule with collapsible formatting
[[module ListPages category="obj" tags="ar_accademia-magia -old_obj" order="titleAsc" rssShow="no" perPage="10" separate="yes"]]
%%linked_title%%
[[/module]]

Can you place this on a page, or below all that on the same page? I'd be interested to see what is on those first two pages.

Also, this might not help, but I've experienced some problems with collapsibles being too close to other content before, try putting an empty line (return) before and after each collapsible module.

Example:

+ Collapsible 1
[[collapsible show="+ open" hide="- close"]]
UNO
[[/collapsible]]
+ Collapsible 2
[[collapsible show="+ open" hide="- close"]]
DUE
[[/collapsible]]
+ ListPagesModule with collapsible formatting
[[module ListPages category="obj" tags="ar_accademia-magia -old_obj" order="titleAsc" rssShow="no" perPage="10" separate="yes"]]
[[collapsible show="+ apri" hide="- nascondi"]]
%%content%%
[[/collapsible]]
[[/module]]
Last edited on 13 Jul 2008 20:38 by David Marseilles
by David Marseilles David Marseilles , 13 Jul 2008 20:37
Re: FIXED
Sulfrum Sulfrum 14 Jul 2008 20:08

I've done the page Fortunzfavor asked me.
The insertion of empty line doesn't change nothing, the bug remain.
Thanks for the feedback, I'll try some other experiments as soon as possible.

here is the page.

Sulfrum


You'll pay for all your sins. If you already paid, please disregard this message.

by Sulfrum Sulfrum , 14 Jul 2008 20:08
Re: FIXED
vaguito vaguito 14 Jul 2008 22:55

Every collapsible block is compiled into two divs whose visibility is exchanged when toggled by the user.

These divs are identified as «collapsible-block-N-folded» and «collapsible-block-N-unfolded», where N is, I believe, an unique block number.

The problem is this numbering is restarted when the module is evaluated. Therefore, the first pair of pages listed will have the same id as those collapsibles outside the module.

I think there's not much you can do by tweaking the wiki page source.

This behaveour also arises when collapsibles are both on the sidebar and the main content. This is actually different, a collapsible on the sidebar may control one in the main content but cannot work alone.

Footnotes
. Note: This word is possibly badly spelled.
Last edited on 17 Jul 2008 18:31 by vaguito
by vaguito vaguito , 14 Jul 2008 22:55
Re: FIXED
Gabrys Gabrys 15 Jul 2008 19:49

Just delete separate=yes.


Piotr Gabryjeluk
visit my blog

by Gabrys Gabrys , 15 Jul 2008 19:49
Re: FIXED
vaguito vaguito 16 Jul 2008 01:53

It seems that is not helping. Have a look at this test page: http://desertline.wikidot.com/sandbox:test-collapsible

by vaguito vaguito , 16 Jul 2008 01:53
Re: FIXED
vaguito vaguito 19 Jul 2008 16:44

Here is another test page, http://desertline.wikidot.com/sandbox:test-collapsible-2 , headers were reordered just in case.
It looks like separate does not make any difference.

I think this, starting from post #post-219591 by Sulfrum, should be moved to another thread with a more suitable title…like "collapsible blocks' id clashing under certain conditions".

Last edited on 19 Jul 2008 19:57 by vaguito
by vaguito vaguito , 19 Jul 2008 16:44
Re: FIXED
cold_blood3d cold_blood3d 29 Aug 2008 04:40

while it's much faster, listpages is still not instantaneous when processing a large amount of stuff. Why does it generate the list when someone visits it? This behavior will always cause a delay. Can't it be pre-generated before hand so it loads up instantly when requested?

3-4 seconds waiting is still enough make someone give up and leave.

edit: Now i'm getting a couple of minutes delay on some pages.

Last edited on 29 Aug 2008 18:21 by cold_blood3d
by cold_blood3d cold_blood3d , 29 Aug 2008 04:40
Unless otherwise stated, the content of this page is licensed under Creative Commons Attribution-Share Alike 2.5 License.
Click here to edit contents of this page.
Click here to toggle editing of individual sections of the page (if possible). Watch headings for an "edit" link when available.
Append content without editing the whole page source.
Check out how this page has evolved in the past.
If you want to discuss contents of this page - this is the easiest way to do it.
View and manage file attachments for this page.
A few useful tools to manage this Site.
Change the name (also URL address, possibly the category) of the page.
View wiki source for this page without editing.
View/set parent page (used for creating breadcrumbs and structured layout).
Notify administrators if there is objectionable content in this page.
Something does not work as expected? Find out what you can do.
General Wikidot.com documentation and help section.
Wikidot.com Terms of Service - what you can, what you should not etc.
Wikidot.com Privacy Policy.

AltStyle によって変換されたページ (->オリジナル) /