2.10 State Redirects and Token errors

[thyon]

I'm getting lots of token errors and also when I save (can't quite know when), it happens that the url adds &state=0 and then redirects to &state=1 and then repeats this until the browser detects the loop... Then you can't go back or do what you did, because that also generates a token error.

Anyone any ideas? So far, 2.10 is a thumbs-down for stability for me...

[Ruud]

I reported the way the token system works as an issue here: http://dev.contao.org/issues/3214 But it got the status invalid for some reason I do not agree with.

The first problem you describe I have not yet encountered, how can I try to reproduce that?

The second problem about the tokens being invalid is something I think is a serious issue. To me a customer has the right to send the same form twice. The best the system may do (Contao, the browser or both if need be) is ask for confirmation and then accept that. Contao 2.10 just shows an error message and all your input data is lost. Clients may not know if their data has been transmitted if something went wrong.

The client is also the most important part of a website; they should never see error messages if it could be avoided. Therefor the token system should never generate unrecoverable error messages like it does.

I opted for the tokens to be stored in the database, and clear each token after x time or first usage. That way a form will have been sent or open for a very long period. It will solve the problem of sessions that get destroyed while containing valid unused tokens. But I got no response to that.

I am actually glad someone else is having problems with the token system. Hoping it will get improved!

You can disable the token system for now. I did...

[Medianomaly]

Wow, I haven't tried 2.10 yet but this sounds discouraging.

Ruud, if that's true I completely agree with everything you've said.

Hopefully this can get worked out, but it sounds like I'll be sticking with 2.9 for now... At least it can be disabled, though...

P.S. -> I guess I should get used to reading a lot of "invalid token" forum posts too...

[ernestmcd]

You can disable the token system for now. I did...

Hi Ruud!

I'm having the same issues with the token errors. Right now the replacement for the formerly NOT error prone system does not seem to work. Contao 2.10.0 probably should not be installed on production sites as this time, and users should consider not upgrading clients until this issue is repaired and tested, and the backup extension is made compatible for 2.10.0.

If I install a 2.9.5 site and then do a manual upgrade to 2.10.0, the site seems to work. If I backup the database and the site, and reinstall in a new directory (clone it basically), the site is functional, I can get to the login for Contao, screen, but the request token error appears and I can't proceed from there.

How did you disable the token request? I've tried the guidance in the Wiki but I guess I don't really understand how the syntax should be, which file is to be edited, and are there any other steps to this?

I'm trying to setup a new demo site on a new server, and so far the token request system is barely functional IMHO.

Ernest McDermon
Snellville, GA

[ernestmcd]

Okay, got it figured out, it took a while to find something that gives the right syntax:

system/config/localconfig.php

Inserting this line immediately following the "allowedTags" Global:

$GLOBALS['TL_CONFIG']['disableRefererCheck'] = true;

I loaded the modified file to the server, pressed [Ctrl] + F5 to refresh the screen, and was able to login to the back end of the website without the error appearing. The link to the solution is at this URL:

http://svn.contao.org/trunk/system/config/config.php

Hopefully Leo will take a look at this and get something resolved on it.

Ernest McDermon
Snellville, GA

[ernestmcd]

I reported the way the token system works as an issue here: http://dev.contao.org/issues/3214 But it got the status invalid for some reason I do not agree with.

I reported it also and got the same response..."Invalid". I got this response last year when an upgrade to Contao resulted in new events being duplicated once in the calendar. One day event appeared on two consecutive days, two day event on four, etc. Eventually another developer in Europe was able to reproduce it and it got fixed in a subsequent Contao release.

The code team will eventually figure out that this IS a problem, and that this many people using Contao 2.10.0 in separate countries and seeing the same behavior cannot all be wrong. Right, I'm disabling the token request so that at the least new platform is workable.

Ernest McDermon
Snellville, GA

[xchs]

I'm having the same issues with the token errors. Right now the replacement for the formerly NOT error prone system does not seem to work. Contao 2.10.0 probably should not be installed on production sites as this time, and users should consider not upgrading clients until this issue is repaired and tested, and the backup extension is made compatible for 2.10.0.

If I install a 2.9.5 site and then do a manual upgrade to 2.10.0, the site seems to work. If I backup the database and the site, and reinstall in a new directory (clone it basically), the site is functional, I can get to the login for Contao, screen, but the request token error appears and I can't proceed from there.

Sorry, but I have to say, that many of the problems reported here (and on the German board) are (still!) mainly user faults or a result of not reading the official announcements and not related to this specific Contao update!

"Invalid request token" notice:

• Read the suggestions regarding the new Request Token System: http://www.contao.org/news/items/contao-2_10_RC1.html[/*:m:1tihwbak]
• Open the folder "templates" (or the back end module "Templates") and check if there are outdated/obsolete templates. For all of these templates YOU are fully responsible! You may have to check for the right file extension (*.html5 / *.xhtml) as well as if there are templates with HTML forms where you have to add the corresponding line regarding the request token.[/*:m:1tihwbak]
• If you are using Third-party Contao extensions which by now are still tagged as not compatible to the latest version 2.10 (you have been warned in the official announcement!) then you have to bother YOURSELF to check whether there are necessary any adjustments regarding the things already said before.[/*:m:1tihwbak]

"Page not found" notice:
[list][*]Open the back end module "Site structure" and check

1. if there is a page of type "Website root" and[/*:m:1tihwbak]
2. if all other (regular) pages are under/within this website root[/*:m:1tihwbak]

[/*:m:1tihwbak][*]Edit the page settings of the page of type "Website root" and check the field "Domain name". If there is already entered a domain name, then consider that it is a difference if you enter a third-level domain name (e.g. all "www" domains) or just a second-level domain name (without "www" and/or a subdomain name).[/*:m:1tihwbak][/list:u:1tihwbak]

How did you disable the token request? I've tried the guidance in the Wiki but I guess I don't really understand how the syntax should be, which file is to be edited, and are there any other steps to this?

You do not have to edit a file manually, if your Contao installation is set up right. You can toggle the mentioned option in the back end module "Settings" -> "Security settings" -> "Disable request tokens" (but keep in mind: this is a potential security risk!)

If the setting of this option is not saved you have a problem with write/access permissions. In such a case you have to configure the so called "Safe Mode Hack" (SMH).

HTH

[Ruud]

Sorry, but I have to say, that many of the problems reported here (and on the German board) are (still!) mainly user faults or a result of not reading the official announcements and not related to this specific Contao update!

Well, in short; I've done all you suggest and submitting a form twice generates an invalid request token message. I fully understand that this is correct, but the problem is that it is my clients and their visitors who do NOT understand. Contao should simply handle the error and process the form OR create a message that will be displayed with the form. Thus allowing to re-submit one way or another.

The color, style and text of the current message do not matter because the way the error is handled is wrong when seen from the website visitors point of view; they do not understand what happened.

This is just one example. If there are more ways to generate a request token error, then the above goes for those messages too.

[Ruud]

I'd like to clarify my problem with the new system:

Suppose I submitted a form, but from the confirmation page I click back because I forgot to mention something in the contact form. After pressing back I can still see my message which I edit and re-submit. Now a screen opens and says:

Invalid request token!

The request token could not be verified. Please go back and try again.

This error occurres if there is a POST request without a valid authentication token. In Contao 2.10, the referer check has been replaced with a request token system. If the problem persists, you are maybe using an incompatible third-party extension or have not correctly updated your Contao installation.

For more information, visit the Contao FAQ page or search the Contao forum.

You have got to be kidding me! I showed this message to my friends with an affinity for computers and they had no clue what this meant!!

I know I can change that message and have it say whatever I want. I can make it look like part of my website, sure... But then still; I went back to do something perfectly ok, to be presented with a "No can't do!".

Let me be clear, concise, and to the point; and not repeat; restate and reiterate the same thing over and over again: I feel the way the error is handled should be changed so that even my mother will understand what happened and what she should do next.

(Yes, I was quoting Mojo Jojo, hope no-one minds )

[xchs]

Hi Ruud,

my post above was not related to you. I'm sure you know all the things I have mentioned.

And yes, I know the problem you mention here and in the ticket system. You can even reproduce this behavior on Leo's site, too: http://www.inetrobots.com/contact-us.html

[attachment=0:ywlqwmx2]token.png[/attachment:ywlqwmx2]

[Ruud]

Hi xchs,

I was not sure, so just to be clear I posted what I think is the problem.

A few weeks ago I even encountered a token problem in the extension repository; it prevented me from updating. The bug causing that has been removed now...

I'd sleep better if Leo would at least reconsider looking at the error handling of the token system...

[Ruud]

Another way to get the invalid request token page: when a form is submitted and I have to wait for the next page to load I could press the button twice, either on purpose or by mistake. This will also show the page.

I know I can make sure this does not happen using javascript.

Again, I understand submitting twice is not nice, and this specific case would be a real logic problem because it is unclear what happened.

[ernestmcd]

my post above was not related to you. I'm sure you know all the things I have mentioned. And yes, I know the problem you mention here and in the ticket system. You can even reproduce this behavior on Leo's site, too: http://www.inetrobots.com/contact-us.html [attachment=0:1ppo8cs9]token.png[/attachment:1ppo8cs9]

Thanks for your previous post about the possible solutions. I've built several dozen Contao websites of varying complexity for our clients and have only just now run into this issue with the new release. I stand with Ruud on this: an error message that appears due to the token system is something that my clients will NOT understand, nor will they pay me to fix this issue. They'll want me to "warranty" the fix, which may mean that the token failure will STILL appear.

For the time being, I'm turning the token system "off" in Contao 2.10.0 installs. I like the idea of enhanced security, but the benefits of the token evaporate when the token error appears continually.

When I submitted this to Leo on the Developer Ticket site, I was told that it could not be reproduced on his site, but obviously you reproduced it. Thanks for taking the time to do that!

Ernest McDermon
Snellville, GA

[leo]

but obviously you reproduced it.

Not quite. He only proved that the token system works like intended and prevents forms from being sent multiple times.

[ernestmcd]

Hi Leo!

I appreciate the comeback, but in my experience, a website that has a contact form which, when submitted for any reason, displays an error that locks up the system and prevents the ordinary visitor from submitting the form, has a serous challenge. I don't see this behavior on other websites with a variety of very sophisticated Content Management Systems. If I make an error on the submission, I simply back up and try again, but I'm not locked from proceeding and I don't get a screen displaying a code error.

On the Google site, if I'm editing my Ad Words or Webmaster tools account, the CAPTCHA routine that Google uses is very difficult for me to use due to the distortion of the letters and numerics that their CAPTCHA uses. I have had to submit a form 4 times in sequence to get it to complete. I never see a lock out on that system, and I'm able to proceed with getting my updates completed.

What I'm seeing here in the USA, is that if I backup a website that's been upgraded to 2.10.0 into a new directory on a current version Linux/Apache blade (e.g. change the directory from a .com to a .org domain), the admin login form on the back end of Contao fails on the first try and displays the token error. Right now I would have to say that the token request routine is unpredictable at best when compared to the behavior of Contao 2.9.5. The warning and lockout is very troublesome IMHO.

I'm deeply appreciative of your efforts to provide the wonderful Contao CMS that I use with our clients every day. All I'm saying is that the goal of stopping visitors to a website from submitting a form more than once, twice, or even several times, may not be the best path forward when dealing with flawed human beings who make common mistakes when using the Internet or a new website, or clicking a form when they want to contact the website for business, buy a product, or whatever.

I just think that there has to be a better way to implement this so that we're not running into visible token errors that preclude even the Administrator from logging in to the Contao back end without taking direct action. A customer or visitor to a 2.10.0 website on seeing the token error will probably just leave and not come back to my clients' websites.

Again, thanks for including a method to disable the token system, that is most helpful right now to deploy the new version of Contao which I'm very excited about :D and actively promoting to our clients.

Hope this helps. Have a great day!

Ernest McDermon
Snellville, GA

[Ruud]

but obviously you reproduced it.

Not quite. He only proved that the token system works like intended and prevents forms from being sent multiple times.

I agree, and sending the same form twice might have unexpected implications and must be prevented if possible and sensible.

Leo, I think you do not understand what the actual problem is. I tried to explain it once, but you did not address the concerns and seemingly ignored whatever I wrote down after your first reply. I understand that with Contao a lot of people may be trying to get your attention so I have tried it again in this thread: viewtopic.php?f=6&t=3378#p14463

Please read that and all replies (except that one off-topic remark). I'm just curious how you respond to my exact problem with the token system's way of handling errors.