Created on 10 Aug 2020 ;    Modified on 10 Aug 2020 ;    Translationitalian

How manage dynamically validation of a field in WTForms

Introduction

In section Covid of this blog [1] we have added the possibility to select the time interval of the data that the user wants to observe.

To do this we have expanded the SelectForm form used to implement in the WEB browser the user's choices. These will be passed on to the application in the WEB server. Which, in turn, will process them by executing the appropriate algorithms that will construct the expected answers. Answers that finally the server will transmit to the user's WEB browser.

Speaking of user entered dates, we need to check that the values received at the server are reasonable [2].

So, among other things, we want to check that the dates entered as the beginning and end of the time interval under observation are among those for which we have the data [3].

How reported in an other article, this blog section is developed using framework Flask.

Flask documenters suggest to use WTForms to manage user's form, something that we do scrupulously.

WTForms is a Python library that simplifies a lot the development and maintenance of forms for WEB applications. In summary: it allows us to design a form using a class whose attributes represent the fields available to the user. There are fields of different types, which cover most form design needs.

For example let's take the following fraction (two fields) of SelectForm from the forms.py module:

class SelectForm(FlaskForm):
    ... <cut> ...

    first   = DateField('from', validators=[InputRequired()], format='%Y-%m-%d')
    last    = DateField('to',   validators=[InputRequired()], format='%Y-%m-%d')

    ... <cut> ...

Fields first and last will contain start and end dates (we note the use of the DateField class) of the interval that the user wants to observe. In the form, the related labels will be from and to. With the validators parameter we hooked the InputRequired control; this requires that the field is filled in, it cannot be empty. Finally, with the format parameter, we have indicated the expected format for the date: year-month-day.

Among other things, one of the main reasons to use a library like this, consists precisely in the fact that usually we find already implemented what is necessary to manage the controls on the contents of the fields sent from the user to the WEB application. This is a particularly critical aspect, as it regards:

  • security considerations [4];
  • and usability aspects of the application, namely the infamous User Interface.

The problem

Indeed, if we consult the WTForms documentation about validators, i.e. controllers of the contents of the fields, we find a page which shows us which controls are already available, and how to write controls customized for our purposes.

So far so good. But in the documentation examples the validators are hooked to the field in the form definition: do we remember InputRequired from the previous example?

In our case the use of this method is not effective because the (custom) control must be able to use as a reference values that change day by day. And unfortunately we are unable to find these values ​​before instantiating the form [5].

So we need to hook our control to the date fields after the form has been instantiated. Assuming you have the TimeRange class that it implements the validator we need, in the views.py module we can do as follows:

def select():
    ... <cut> ...

    form = forms.SelectForm()
    ... <cut> ...

    if request.method=='POST':
        time_range1 = forms.TimeRange(FIRST, LAST)  #+- initializing TimeRange validator for POST
        form.first.validators.append(time_range1)   #+- binding dynamically timerange validator to field first
        form.last.validators.append(time_range1)    #+- as above to field last

    if form.validate_on_submit():
        ... <cut> ...
        return redirect(... <cut> ...)

    ... <cut> ...
    return render_template(... <cut> ...)

That is: we instantiate the form, we upload any dynamic data, and, if we need [6], we instantiate the TimeRange control and append it to the list of validators in the fields first and last.

If the form passes the checks of the validate_on_submit we do our own processing and with the redirect we go to the application logic it implements the required commands.

If the form does not pass the checks we send it back to the WEB browser with the render_template.

With this we might think we have done it. But it is not. Unfortunately, with this version of WTForms if we dynamically hook validators, these remain available even after the destruction of the form; and reused in subsequent calls, despite having been created new instances. This behavior can expose us to intermittent malfunctions, the cause of which is difficult to identify [7].

So we have to take care of a proper clean-up. We can expand the above code as follows:

def select():
    ... <cut> ...

    form = forms.SelectForm()
    ... <cut> ...

    if request.method=='POST':
        time_range1 = forms.TimeRange(FIRST, LAST)  #+- initializing TimeRange validator for POST
        form.first.validators.append(time_range1)   #+- binding dynamically timerange validator to field first
        form.last.validators.append(time_range1)    #+- as above to field last

    if form.validate_on_submit():
        ... <cut> ...

        form.first.validators.remove(time_range1)  # we need to remove timerange validators to destroy them
        form.last.validators.remove(time_range1)   #    otherwise they will be reused in succedings calls

        return redirect(... <cut> ...)

    try:                                           # again: removing timerange validators. see previous comment
        if time_range1 in form.first.validators:
            form.first.validators.remove(time_range1)
        if time_range1 in form.last.validators:
            form.last.validators.remove(time_range1)
    except:
        pass

    ... <cut> ...
    return render_template(... <cut> ...)

The TimeRange validator

Just to complete the speech, below we report the essential to implement the TimeRange validator:

class TimeRange(object):
    ... <cut> ...
    def __init__(self, first=None, last=None, message=None):
        if not first:
            first = date.today()
        if not last:
            last = date.today()
        if not message:
            message = 'Field must be between {} and {}'.format(first.strftime('%Y-%m-%d'), last.strftime('%Y-%m-%d'))
        self.first = first
        self.last  = last
        self.message = message

    ... <cut> ...

    def __call__(self, form, field):
        fname = 'TimeRange().__call__'
        f = field.data and field.data<self.first
        l = field.data and field.data>self.last
        if f or l:
            raise ValidationError(self.message)

Conclusioni

To implement a dynamic custom validator to use with WTForms is an affordable task. But it is necessary pay attention to instance management. Activity that usually in Python is neglected, thanks to the excellent functioning of garbage collector which usually relieves the programmer of this burden.

Enjoy. ldfa


[1]The complete code of this project is readable at thi GitHub address.
[2]

There are two (three) different ways of doing these checks. We can

  • check in the WEB browser, before sending the data to the server; it is the most user-responsive way, but it must be implemented with javascipts present in the form, which weigh up it a little (I am referring to the size of the HTML page transmitted);
  • check in the WEB server when arrive the user choices, made in the WEB browser; a less responsive method, but the HTML pages are smaller, and we're safe even in case any malicious people modify the data transmitted from the WEB browser to the server.

The third way is a hybrid of the above: AJAX; it consists of javascripts that send a single selection to the server at a time; and server responds on the fly and selectively, i.e. does not send the whole page, but only updating of the field being manipulated.

However, this method does not exempt us from carrying out checks on the WEB server when the entire page is sent at the conclusion of the choices made by the user. Because there is always the possibility that "someone" modifies the data transmitted by the WEB browser to the server.

[3]Time interval of available data is written in the 4th paragraph of the presentation page of the section.
[4]Security because these fields can be maliciously used to launch attacks on the application. Hence the importance to control their content.
[5]

At least: the writer was unable to identify these values in the module forms.py because from inside it cannot import views.py. See the project for related details.

Now it occurs to me that I could have moved the form definition from the forms.py module to the views.py module, which hosts the application logic. Does this seem to you an elegant solution? I do not like it: it takes us towards a monolithic code.

[6]We will have to check the user's choices, so we load the validator after the choices have been made: i.e. when we receive them with a POST.
[7]We speak by direct experience. The #2 malfunction of this project took us a lot of effort. We thought we solved it at least three times, to see it reappear the day after the last fix. To understand what was happening we had to log in allocation addresses of instances to observe their use by the application.