history state (due to pushState) issues

oferlaor
User w
User w
Posts: 10
Joined: Wed Jan 27, 2021 2:24 pm

history state (due to pushState) issues

Post by oferlaor »

Hi,

I've been struggling to debug a strange behavior that the plugin exhibits.

How to reproduce on this forum:
1. Scroll in the forum list to the bottom
2. click the last forum (e.g., Apache and Server related)
3. Press back on the browser - it jumps to the correct location, but to the top of the page and note that the URL remains the same as the previous page (not the list of forums).
4. Press back on the browser again - NOW it jumps both to the correct location on the page and the correct URL.

I've been able to track down what I think is the root cause. It looks like when you click, the code in the phpbb overall_footer will notice that it is an internal call and convert the A-click into a message to the parent of the IFRAME.

$(document).on("click", "a", function(e) {
......
if ('parentIFrame' in window){
if( typeof w3allNOappend == 'undefined' || w3allNOappend == false ){
window.parentIFrame.sendMessage(w3allappend);
}
}

The message of the IFRAME does the pushState with the correct URL. However, there are now two events in the system - pushState with the correct state and the original A-click (seen above) which doesn't preventDefault.

I tried a few things, but no luck. I tried replaceState so that the state would override the A click's state, but that doesn't work (I think because theres' not always a state to replace). I also tried to preventDefault on the A tags' click event. No luck. The only thing that works is to change the location of the root of the iframe - but that's really inefficient as it loads everything from the beginning.

ideas or suggestions?

User avatar
axew3
w3all User
w3all User
Posts: 2230
Joined: Fri Jan 22, 2016 5:15 pm
Location: Italy
Contact:

Re: history state (due to pushState) issues

Post by axew3 »

so you may mean this, i try to explain:
when you land into page forum, no history is pushed then what you say happen. The push happen on click, when phpBB footer code response, and only in this case (for some click event, response is avoided on phpBB footer code as you saw, may something more need to be adjusted on it, we'll see). But this is not the problem in this case maybe.

To try a fix for this on fly test, i've do this:

open page-forum.php (or whatever you named it) into your active template folder
and search for this code on the very bottom:

Code: Select all

</script>";
?>
</div>
<!-- END iframe div -->
<?php get_footer(); ?>
just before the closing </script> tag, add the follow:

Code: Select all

w3allpurl = '/wordpress/forum';
history.pushState({w3allpurl: w3allpurl}, \"Forums\", w3allpurl);
note that you have to adjust
/wordpress/forum
to correctly point to your page forum in wordpress, test by reloading the page and see what url is pushed and you'll see if you're correct with this setting.
Now the index is correctly pushed and when you go to return back then it will be loaded into address bar.

This example anyway, as i propose you, works if you land pointing to the index: if you point to another page, like ex
https://subdomain.w3host.com/phpbb3/viewforum.php?f=2
it will be by the way pushed as index page: it is necessary to push any landed page rebuilding correct pairs
And to do this, the correct url need to be rebuilt to be pushed as base64encoded and with proper fancy url.

Thank you for report, it will be fixed on v6 code if you do not return here with a code solution.

Have i forgot or misunderstand something?

Guest

Re: history state (due to pushState) issues

Post by Guest »

1. this is an example of how
https://www.youtube.com/watch?v=gVQrm3x ... e=youtu.be

2. BTW, we've been able to fix some of the issues your users face:

a. reloading the page after login (i.e., the typical scenario is that a user jumps to a part of the page, gets logged in automatically and it causes the forum to jump to the top index rather than the correct location (his original request). This is particularly annoying when users jump to an unread post and get into the home page. Also, note that internally, phpbb sometimes logs you out and back in implictly (if you're logged in multiple computers and the session timeout expired), so this fix is very useful.

b. Another issue is the email to unread. When the user gets the link, he opens up the URL, this marks the topic as read, and the header will reload the image, causing the page to redirect to the WP wrapper, but losing the #unread. Our fix is ugly and requires modification of phpbb core files, basically we encode the base64 call so that there's no need for the redirect

c. Users Logging in too many times. This is a problem with phpbb. There's a stupid verification key that lives inside the forms, and they validate origin IP, time, and many other items. I'm not really sure what this is used for, but if you're behind a CDN, have smart caching, or your users use mobile phones, their IP and other properties can fail and users either get logged in too many times, or they actually get rejected when they try to post. There are many settings in the default phpbb configuration that are much too harsh for large scale normal situations. just had an "submitted form was invalid" error on this forum because uploading the video took a while. Ironic.

To see what we've done, you can look at the modified code here: https://htmag.co.il/forum

User avatar
axew3
w3all User
w3all User
Posts: 2230
Joined: Fri Jan 22, 2016 5:15 pm
Location: Italy
Contact:

Re: history state (due to pushState) issues

Post by axew3 »

Great!

About
read/unread email notification
did you applied something like this?
viewtopic.php?f=2&t=1618

Going to check asap, so useful info, and also the site and the template is nice.
Awesome, my compliments!

I will may return to ask you something if something not clear, thank you for the report.
The final solution may will push the url to the address bar.

oferlaor
User w
User w
Posts: 10
Joined: Wed Jan 27, 2021 2:24 pm

Re: history state (due to pushState) issues

Post by oferlaor »

The unread solution that we did ultimately was to change the URL the user receives to the BASE64 encoded method, it required core file change in post.php:

'U_NEWEST_POST' => generate_board_url(true) . "/forum?q=" . base64_encode("viewtopic.{$this->php_ext}?f={$this->get_data('forum_id')}&t={$this->item_parent_id}&e=1&view=unread#unread"),

The unread trick you did there is to detect which items were unread and then using that for the redirect point. The scrolling would work, but the items wouldn't be marked as new (we actually change the shade a bit to make it more apparent).

A cleaner way to do it, is to modify htaccess to redirect "view=unread" and redirect to an endpoint that would encode it. One could also modify the notification email text template to point to that unread end point.

User avatar
axew3
w3all User
w3all User
Posts: 2230
Joined: Fri Jan 22, 2016 5:15 pm
Location: Italy
Contact:

Re: history state (due to pushState) issues

Post by axew3 »

The js unread trick above, just let the code follow to the overall_footer code, if the request is for unread:
at the time the page load, then you get the last unread and redirect exactly to this post.
Since normally this is what happen on default behavior, it is exactly what is expected as result.
Yes the htaccess is more consistent and offer several benefits under many aspects. The first is the page reload that you can avoid to happen.

The old htaccess concept of the v1 and v2 code deserve to be improved and it will be.
It is long time that i think to have ready snippets for both js or htaccess.

v6 code will contain several improvements, also based on what we discussed and you suggested here.

i'm over these days on finish with Audio/Video recorder into phpBB posts extension.
So we can start to use the phpBB forum as never we had think it could be.
Easy, unbelievable light, allowing to record mp3 for audio and webm for video, that can be recorded directly on post.
You can listen the result, then just click to insert the audio or video into the post as attachment.
It will be possible to play and listen each single audio, but also a button to listen/play all topic's audio video sequentially, like you would do into a telegram/whatsapp message will be provided (that's quite easy as concept: you'll not believe that all this would be possible without changing/adding nothing into phpBB events and code). phpBB contain all the necessary by default.
I'm sure that many will love it.

About video:
Unfortunately, webm is still not completely supported on ios safari?
Time ago i did a video/audio recorder plugin for wordpress (it is still working fine here on my localhost under last wp5.6 and even php8, but i never released as free, i will do when asap i will have the time to review the code i did years ago), and for video i had to install ffmpeg on server to convert streams into mp4, the unique video format supported by all browsers.
Well, the extension ideally will provide both solutions: more advanced could install and use ffmpeg, but the default will be webm.
Just charging my old ipad, i will update and test things on it to check where we are with webm on safari in 2021.

p.s yes checked, webm is still unsupported into ios: tested firefox and safari, both not working, webm is not recognized

Will be possible to use only for video/audio, or only audio, or both. It will be awesome, and as said, unbelievable easy and light.

https://caniuse.com/?search=webm


Post a new topicREC1.png
Post a new topicREC1.png (13.49 KiB) Viewed 892 times
videoAudioToPost.png
videoAudioToPost.png (13.06 KiB) Viewed 892 times

Guest

Re: history state (due to pushState) issues

Post by Guest »

thinking about this problem over the weekend:

1. the current flow is that the IFrame basically changes location, right before it does - it sends the message to the parent. The parent does pushState, so you effectively have two states - the state actually generated by the click event and the second one that was created by the pushState.

2. I thought of a stupid workaround, basically hooking into the popState event and going back one extra time if it sees it was originally pushed by the code (pretty easy). The problem is that it looks like the popState event only gets called during the second (good) back command is issued. Still thinking about how to approach it.

3. This is what someone else did (it's a bit convoluted, basically removes the original request, creates a new cloned iframe off-DOM and then reconnects it, so it doesn't get entered into the history stack).

https://stackoverflow.com/questions/397 ... wo-entries

User avatar
axew3
w3all User
w3all User
Posts: 2230
Joined: Fri Jan 22, 2016 5:15 pm
Location: Italy
Contact:

Re: history state (due to pushState) issues

Post by axew3 »

i will try to dedicate something today if possible, to fix this and the loop: substantially to me, the suggested way about this in my previous post, is the unique necessary: assign the correct url when you land into wp page forum, which do not happen now: so if you land into a topic, then you follow navigating, then you return back, you'll return back only until the first click, because the one of where you landed, isn't pushed in the code as is.
The other aspect to check, is just the overall_footer added code: is it avoiding to pass the value to parent, when instead should do?

These aspects i will check only, i'm sure these are the unique things we'll have to fix.

User avatar
axew3
w3all User
w3all User
Posts: 2230
Joined: Fri Jan 22, 2016 5:15 pm
Location: Italy
Contact:

Re: history state (due to pushState) issues

Post by axew3 »

So tested the think, and all should now be fixed. A more deep test can confirm.
The code is running also into this online, just updated the page-forum with this last version here in attach.
The problem was that the code, when landing into any page, was not pushing the url to history.
In the attached and modified page-forum that is running right now also here online, has been added the following code on bottom, right before the closing </script > tag:

Code: Select all

 jQuery( window ).load(function() { // wrapped here, or won't push correctly
  var w3all_landed_url = '".$w3all_url_to_cms."';
  var w3all_landed_url_clean = w3allNormalize_phpBBUrl_onParent(w3all_landed_url);
  // PUSH phpBB URLs when wp page load first time //
  var w3all_passedurl = window.btoa(unescape(encodeURIComponent(w3all_landed_url_clean)));
  var w3all_passedurl_push = '".$w3allhomeurl."/".$wp_w3all_forum_folder_wp."/?".$w3all_iframe_custom_w3fancyurl."=' + w3all_passedurl;
  history.pushState({w3all_passedurl: w3all_passedurl_push}, \"\", w3all_passedurl_push);
  jQuery('#w3_toogle_wrap_loader').attr( \"class\", \"w3_no_wrap_loader\" );
 })
note that the
jQuery( window ).load(function() { instruction wrap the code: if you remove, the url is NOT correctly pushed to history.
Also note that i copied/cloned the w3allNormalize_phpBBUrl function, used on overall_header and overall_footer, and added into page forum on header, ready to be used also here in this context, so on page forum, to push correct values, and i named it w3allNormalize_phpBBUrl_onParent (you can see it on top where js of the header output is).

this is the new page-forum:
page-forum.zip
(5.18 KiB) Downloaded 28 times

ps: the last jQuery('#w3_toogle_wrap_loader').attr( \"class\", \"w3_no_wrap_loader\" ); line of code, of course is not related this problem, i have just put it there because i still experience the problem on removing the preloader screen sometime, which i have not still well understand why/when it occur). Or to better say, it come out due to possible js errors, both on wp templates or phpBB. A way to resolve this will be applied soon.

p.s your subsequent question is maybe: why if it is not wrapped inside jQuery( window ).load(function() { won't work?
My answer is: i do not know exactly, but as you also argued before, i assumed that the history push was not doing his work correctly may because fired too soon. There are many possible answers for this. But i will not go to confuse you further more with my assertions.
I imagined it could come out due to the fact that the timing when the instruction execute, may goes to conflict with something else or because it was running before something else, that was rewriting the thing in some way. Now it work.

oferlaor
User w
User w
Posts: 10
Joined: Wed Jan 27, 2021 2:24 pm

Re: history state (due to pushState) issues

Post by oferlaor »

The version on this site still exhibits the problem.

Please see here:
https://www.youtube.com/watch?v=dJ9tOCy ... e=youtu.be

The issue is not just on the front forum page, it is on every A href click.

BTW, very cool on the recorder!

oferlaor
User w
User w
Posts: 10
Joined: Wed Jan 27, 2021 2:24 pm

Re: history state (due to pushState) issues

Post by oferlaor »

BTW, you can use the history state to make sure that you're not in a loop. If you see that the last state is the exact same location, you're running in a loop.

User avatar
axew3
w3all User
w3all User
Posts: 2230
Joined: Fri Jan 22, 2016 5:15 pm
Location: Italy
Contact:

Re: history state (due to pushState) issues

Post by axew3 »

The issue is not just on the front forum page, it is on every A href click.
the problem was into any landed page: for landed page, i mean the very first phpBB page loaded due to explicit direct url to a post, a topic or index.php. Now it has been resolved. And return ever the corect result, but not on address bar, ok now it is clear the meaning
Thank you for the video explain, perfect, i will go to check how to resolve asap and return back here!

yes! the recorder will be useful/nice!

User avatar
axew3
w3all User
w3all User
Posts: 2230
Joined: Fri Jan 22, 2016 5:15 pm
Location: Italy
Contact:

Re: history state (due to pushState) issues

Post by axew3 »

ok, i've got the problem.
in short, what happen is that history.pushState, push two times same page (you can see into history back browser button)

if you switch to

Code: Select all

history.replaceState
instead of using history.pushState
it lead to the correct result. But of course, the address bar is not updated then.
same was for the solution on landing, but there, it has been possible to wrap into load jQuery( window ).load(function() {
to resolve.
i will check how could never be possible to fix this-
the good question is: why it push two times the same?

User avatar
axew3
w3all User
w3all User
Posts: 2230
Joined: Fri Jan 22, 2016 5:15 pm
Location: Italy
Contact:

Re: history state (due to pushState) issues

Post by axew3 »

so for what it worth my consideration on this, i just take a look into more deeply.

the resume is this to me (that may you expressed before and may i did not understood) of all the problems.

When you land, the first page is correctly pushed ONCE into history with the fix i applied.

but as soon you click into any forum link, and the history.pushState fire, it cause that the history push happen TWO TIMES, you can see it into the Back Button of the browser: there are two identical pushes, two on history: so, if you do not care to push on the address bar the url when you click into back button, the solution is this:

on the top javascript, so inside the function that output the javascript on page-forum, just add this:

Code: Select all

window.onpopstate = function(event) {
  window.history.go(-1);
};
with this, the back button action will cause the correct return to the right page, even if the address bar will not update to where you return back, then if you reload the page, you'll may be redirected to the url that is on the address bar.
This appear to me at moment the more clean solution, but it is not perfect.

But the fundamental question is and still remain by the way:
why this happen? how it has been possible to be resolved wrapping the code like this, the solution of when you land to the first page on forum:

Code: Select all

 jQuery( window ).load(function() { // wrapped here, or won't push correctly, duplicate happen
  var w3all_landed_url = '".$w3all_url_to_cms."';
  var w3all_landed_url_clean = w3allNormalize_phpBBUrl_onParent(w3all_landed_url);
  // PUSH phpBB URLs when wp page load first time //
  var w3all_passedurl = window.btoa(unescape(encodeURIComponent(w3all_landed_url_clean)));
  var w3all_passedurl_push = '".$w3allhomeurl."/".$wp_w3all_forum_folder_wp."/?".$w3all_iframe_custom_w3fancyurl."=' + w3all_passedurl;
  history.pushState({w3all_passedurl: w3all_passedurl_push}, \"\", w3all_passedurl_push);
 })
the unique thing i note, is that the code that works fine for the first loaded page,
is that this code is NOT inside the callback of the iframe resizer code.
is the iframe resizer code interacting some way with this?
inside it, happen two times
... we'll see

oferlaor
User w
User w
Posts: 10
Joined: Wed Jan 27, 2021 2:24 pm

Re: history state (due to pushState) issues

Post by oferlaor »

did you try the popstate solution?

I thought of it myself but it didn't fire the first back, only on the second...

I think it pushes it twice because you do not preventDefault() on the actual click, so you have one event by the original click and the second is the fake one created by the pushState.

is the go back solution implemented on this forum already?

BTW, found another neat hack - I moved the iframeresizer to the overall header and it really accelerated how fast the forum loads.

User avatar
axew3
w3all User
w3all User
Posts: 2230
Joined: Fri Jan 22, 2016 5:15 pm
Location: Italy
Contact:

Re: history state (due to pushState) issues

Post by axew3 »

yes applied

ps
found another neat hack - I moved the iframeresizer to the overall header and it really accelerated how fast the forum loads.
i will come to see how you did if you do not provide here a ready example!
Provide one when ready if you can, so may it will be so published as default for all (with credits for you, of course)

oferlaor
User w
User w
Posts: 10
Joined: Wed Jan 27, 2021 2:24 pm

Re: history state (due to pushState) issues

Post by oferlaor »

OK, tried it.

But it doesn't quite work correctly (almost). The popstate event gets called the second back (so you lose the scroll position).

Is there a way to catch the first event and do the go back the first time?

User avatar
axew3
w3all User
w3all User
Posts: 2230
Joined: Fri Jan 22, 2016 5:15 pm
Location: Italy
Contact:

Re: history state (due to pushState) issues

Post by axew3 »

will try to go more deep asap ... i go to check your solution about preloader, because of that still stop here, often not correctly removed, i have to reload the page.
Or i will rewrite if still in trouble with, with a different solution.

For what i see into firefox and chrome it return correctly to the scrolled position at moment.

While what i really do not understand is why the preloader do not disappear often ... it is very strange to me, exactly happen when i may click on back button to return to forum on the very first history stack

oferlaor
User w
User w
Posts: 10
Joined: Wed Jan 27, 2021 2:24 pm

Re: history state (due to pushState) issues

Post by oferlaor »

The preloader thing is only a preliminary idea. I haven't been able to get to it yet.

About the history state, I was thinking last night about a possible solution, but tried it this morning and it failed. Still not sure why (this is a very convoluted issue).

popState appears to be triggering only on the second "Back" command, which actually holds the scrolling position. I was trying to find another possible event that could trigger instead. I found a couple but they are mostly page lifecycle events triggered by the move.

Then, I was thinking, what if I could change the order so that the popState happens last, so that the popState would trigger on the first BACK, that would let me do the history.back() inside the popState with no additional side effects.

This is the code I tried, however strangely it does not work. The popState event always happens on the second ("Good") back. I tried adding a setTimeout 0 so that the pushState definitely happens after the new URL registers, but I don't think that works.

Code: Select all

    iFrameResize({
			log         : false,
			inPageLinks : true,
			targetOrigin: '".$w3all_url_to_cms."', 
			checkOrigin : w3all_orig_domains, 
		 // heightCalculationMethod: 'documentElementOffset', // If iframe not resize correctly, un-comment (or change with one of others available resize methods) 
		 // see: https://github.com/davidjbradshaw/iframe-resizer#heightcalculationmethod       
			onMessage : function(messageData){ // Callback fn when message is received 	
				// w3all simple js check and redirects
				var w3all_passed_url = messageData.message.append.toString();
				var w3all_ck = \"".$_SERVER['SERVER_NAME']."\";
				var w3all_pass_ext  = (w3all_passed_url.indexOf(w3all_ck) > -1);    
				var w3all_ck_preview = (w3all_passed_url.indexOf('preview') > -1);

				if (w3all_ck_preview == false) { // or the phpBB passed preview link, will be recognized as external, and preview will redirect to full forum url instead
				 // so these are maybe, external iframe redirects
					if (w3all_pass_ext == true) {
						window.location.replace(w3all_passed_url); 
					 }
					if (/^(f|ht)tps?:\/\//i.test(w3all_passed_url)) {
					  window.location.replace(w3all_passed_url); 
					}
				}
				// do not pass to be encoded an url with sid or if it point to phpBB admin ACP via iframe
				if( /[^-0-9A-Za-z\._#\:\?\/=&%]/ig.exec(w3all_passed_url) !== null || /adm\//ig.exec(w3all_passed_url) !== null || /sid=/ig.exec(w3all_passed_url) !== null ){
					w3all_passed_url = '';
				}
				// PUSH phpBB URLs //
				w3all_passed_url = window.btoa(unescape(encodeURIComponent(w3all_passed_url)));
				w3all_passed_url_push = '".$w3allhomeurl."/".$wp_w3all_forum_folder_wp."/?".$w3all_iframe_custom_w3fancyurl."=' + w3all_passed_url;
				messageData.iframe.contentWindow.document.location.href= messageData.message.ref;
				history.pushState({w3all_passed_url: w3all_passed_url}, \"Forums\", w3all_passed_url_push);
			}
	});
	window.onpopstate = function(event) {
	  console.log(\"pop state: \" + JSON.stringify(event.state));
	};	

Code: Select all

			if ('parentIFrame' in window){
				if( typeof w3allNOappend == 'undefined' || w3allNOappend == false ){
					window.parentIFrame.sendMessage({ append:w3allappend, ref: href});
					e.preventDefault();
				}
			}

Documentation also says replaceState should also cause a popState, but in my testing that does not work either.

The only thing I think might do the trick is to clone the iframe, change its source and then connect it to the IframeResizer and then kill the old one...

I'm out of ideas...

User avatar
axew3
w3all User
w3all User
Posts: 2230
Joined: Fri Jan 22, 2016 5:15 pm
Location: Italy
Contact:

Re: history state (due to pushState) issues

Post by axew3 »

Thank you for all important hints and observations that leaded me to understand the problem.
Giving to this aspect a more deep look into, before the v6 code release, so that will contain all the possible (correct) solutions that in the while will come out

User avatar
axew3
w3all User
w3all User
Posts: 2230
Joined: Fri Jan 22, 2016 5:15 pm
Location: Italy
Contact:

Re: history state (due to pushState) issues

Post by axew3 »

Hello!
just over the iframe code, but at moment nothing has not been changed in the overall_footer.html part.

while page-forum, will be updated to fix the wp uid1 / phpBB2 uid2 loop
this history push bug you reported and the preloader loop

About this problem, on pushState (and duplicated history insertions), i see that it can be resolved like this:

open page-forum, and where this line:

Code: Select all

history.pushState({w3all_passed_url: w3all_passed_url}, \"\", w3all_passed_url_push);
change into this:

Code: Select all

    //history.pushState({w3all_passed_url: w3all_passed_url}, \"\", w3all_passed_url_push);
    history.replaceState({w3all_passed_url: w3all_passed_url}, \"\", w3all_passed_url_push);
more below, there is this code added, to fix the when landing to forum pushState:

Code: Select all

 jQuery( window ).load(function() { // wrapped here, or won't push correctly
  var w3all_landed_url = '".$w3all_url_to_cms."';
  var w3all_landed_url_clean = w3allNormalize_phpBBUrl_onParent(w3all_landed_url);
  // PUSH phpBB URL when wp page load first time //
  var w3all_passedurl = window.btoa(unescape(encodeURIComponent(w3all_landed_url_clean)));
  var w3all_passedurl_push = '".$w3allhomeurl."/".$wp_w3all_forum_folder_wp."/?".$w3all_iframe_custom_w3fancyurl."=' + w3all_passedurl;
  history.replaceState({w3all_passedurl: w3all_passedurl_push}, \"\", w3all_passedurl_push);
  jQuery('#w3_toogle_wrap_loader').attr( \"class\", \"w3_no_wrap_loader\" );
 })
there are two more little fixes about id1 uid2 and the empty fancy or w3= empty, and it is all working fine now into my tests.

this is the page forum that: fix uid1 WP and phpBB uid2 loop
fix the pushState on page forum landing and oniframe navigation
fix the empty w3= (or custom fancy= url) that sometime was happening on address bar

fix the preloader that sometime do not disappear (and there is no stop until page reloaded may do to explicit reload)?
not totally sure, moments ago i realized on mind all the flow, but not focused if tricks added really fixed also this aspect.

this new patched file has been applied here online, so you can see how it work now, that seem to be the right way.
Let know!
Attachments
page-forum.zip
(5.22 KiB) Downloaded 24 times

Post Reply