Defining the Domain Model

The first change we'll make to our stock cookiecutter-generated application will be to define a wiki page domain model.

Note

There is nothing special about the filename user.py or page.py except that they are Python modules. A project may have many models throughout its codebase in arbitrarily named modules. Modules implementing models often have model in their names or they may live in a Python subpackage of your application package named models (as we've done in this tutorial), but this is only a convention and not a requirement.

Declaring dependencies in our setup.py file

The models code in our application will depend on a package which is not a dependency of the original "tutorial" application. The original "tutorial" application was generated by the cookiecutter; it doesn't know about our custom application requirements.

We need to add a dependency, the bcrypt package, to our tutorial package's setup.py file by assigning this dependency to the requires parameter in the setup() function.

Open tutorial/setup.py and edit it to look like the following by adding bcrypt and sorting the packages:

11requires = [
12    'alembic',
13    'bcrypt',
14    'plaster_pastedeploy',
15    'pyramid',
16    'pyramid_debugtoolbar',
17    'pyramid_jinja2',
18    'pyramid_retry',
19    'pyramid_tm',
20    'SQLAlchemy',
21    'transaction',
22    'waitress',
23    'zope.sqlalchemy',
24]

It is a good practice to sort packages alphabetically to make them easier to find. Our cookiecutter does not have its packages sorted because it merely tacks on additional packages depending on our selections.

Note

We are using the bcrypt package from PyPI to hash our passwords securely. There are other one-way hash algorithms for passwords if bcrypt is an issue on your system. Just make sure that it's an algorithm approved for storing passwords versus a generic one-way hash.

Running pip install -e .

Since a new software dependency was added, you will need to run pip install -e . again inside the root of the tutorial package to obtain and register the newly added dependency distribution.

Make sure your current working directory is the root of the project (the directory in which setup.py lives) and execute the following command.

On Unix:

$VENV/bin/pip install -e .

On Windows:

%VENV%\Scripts\pip install -e .

Success executing this command will end with a line to the console something like the following.

Successfully installed bcrypt-3.2.0 cffi-1.14.4 pycparser-2.20 tutorial

Remove mymodel.py

Let's delete the file tutorial/models/mymodel.py. The MyModel class is only a sample and we're not going to use it.

Add user.py

Create a new file tutorial/models/user.py with the following contents:

 1import bcrypt
 2from sqlalchemy import (
 3    Column,
 4    Integer,
 5    Text,
 6)
 7
 8from .meta import Base
 9
10
11class User(Base):
12    """ The SQLAlchemy declarative model class for a User object. """
13    __tablename__ = 'users'
14    id = Column(Integer, primary_key=True)
15    name = Column(Text, nullable=False, unique=True)
16    role = Column(Text, nullable=False)
17
18    password_hash = Column(Text)
19
20    def set_password(self, pw):
21        pwhash = bcrypt.hashpw(pw.encode('utf8'), bcrypt.gensalt())
22        self.password_hash = pwhash.decode('utf8')
23
24    def check_password(self, pw):
25        if self.password_hash is not None:
26            expected_hash = self.password_hash.encode('utf8')
27            return bcrypt.checkpw(pw.encode('utf8'), expected_hash)
28        return False

This is a very basic model for a user who can authenticate with our wiki.

We discussed briefly in the previous chapter that our models will inherit from an SQLAlchemy sqlalchemy.ext.declarative.declarative_base(). This will attach the model to our schema.

As you can see, our User class has a class-level attribute __tablename__ which equals the string users. Our User class will also have class-level attributes named id, name, password_hash, and role (all instances of sqlalchemy.schema.Column). These will map to columns in the users table. The id attribute will be the primary key in the table. The name attribute will be a text column, each value of which needs to be unique within the column. The password_hash is a nullable text attribute that will contain a securely hashed password. Finally, the role text attribute will hold the role of the user.

There are two helper methods that will help us later when using the user objects. The first is set_password which will take a raw password and transform it using bcrypt into an irreversible representation, a process known as "hashing". The second method, check_password, will allow us to compare the hashed value of the submitted password against the hashed value of the password stored in the user's record in the database. If the two hashed values match, then the submitted password is valid, and we can authenticate the user.

We hash passwords so that it is impossible to decrypt them and use them to authenticate in the application. If we stored passwords foolishly in clear text, then anyone with access to the database could retrieve any password to authenticate as any user.

Add page.py

Create a new file tutorial/models/page.py with the following contents:

 1from sqlalchemy import (
 2    Column,
 3    ForeignKey,
 4    Integer,
 5    Text,
 6)
 7from sqlalchemy.orm import relationship
 8
 9from .meta import Base
10
11
12class Page(Base):
13    """ The SQLAlchemy declarative model class for a Page object. """
14    __tablename__ = 'pages'
15    id = Column(Integer, primary_key=True)
16    name = Column(Text, nullable=False, unique=True)
17    data = Column(Text, nullable=False)
18
19    creator_id = Column(ForeignKey('users.id'), nullable=False)
20    creator = relationship('User', backref='created_pages')

As you can see, our Page class is very similar to the User defined above, except with attributes focused on storing information about a wiki page, including id, name, and data. The only new construct introduced here is the creator_id column, which is a foreign key referencing the users table. Foreign keys are very useful at the schema-level, but since we want to relate User objects with Page objects, we also define a creator attribute as an ORM-level mapping between the two tables. SQLAlchemy will automatically populate this value using the foreign key referencing the user. Since the foreign key has nullable=False, we are guaranteed that an instance of page will have a corresponding page.creator, which will be a User instance.

Edit models/__init__.py

Since we are using a package for our models, we also need to update our __init__.py file to ensure that the models are attached to the metadata.

Open the tutorial/models/__init__.py file and edit it to look like the following:

  1from sqlalchemy import engine_from_config
  2from sqlalchemy.orm import sessionmaker
  3from sqlalchemy.orm import configure_mappers
  4import zope.sqlalchemy
  5
  6# Import or define all models here to ensure they are attached to the
  7# ``Base.metadata`` prior to any initialization routines.
  8from .page import Page  # flake8: noqa
  9from .user import User  # flake8: noqa
 10
 11# Run ``configure_mappers`` after defining all of the models to ensure
 12# all relationships can be setup.
 13configure_mappers()
 14
 15
 16def get_engine(settings, prefix='sqlalchemy.'):
 17    return engine_from_config(settings, prefix)
 18
 19
 20def get_session_factory(engine):
 21    factory = sessionmaker()
 22    factory.configure(bind=engine)
 23    return factory
 24
 25
 26def get_tm_session(session_factory, transaction_manager, request=None):
 27    """
 28    Get a ``sqlalchemy.orm.Session`` instance backed by a transaction.
 29
 30    This function will hook the session to the transaction manager which
 31    will take care of committing any changes.
 32
 33    - When using pyramid_tm it will automatically be committed or aborted
 34      depending on whether an exception is raised.
 35
 36    - When using scripts you should wrap the session in a manager yourself.
 37      For example:
 38
 39      .. code-block:: python
 40
 41          import transaction
 42
 43          engine = get_engine(settings)
 44          session_factory = get_session_factory(engine)
 45          with transaction.manager:
 46              dbsession = get_tm_session(session_factory, transaction.manager)
 47
 48    This function may be invoked with a ``request`` kwarg, such as when invoked
 49    by the reified ``.dbsession`` Pyramid request attribute which is configured
 50    via the ``includeme`` function below. The default value, for backwards
 51    compatibility, is ``None``.
 52
 53    The ``request`` kwarg is used to populate the ``sqlalchemy.orm.Session``'s
 54    "info" dict.  The "info" dict is the official namespace for developers to
 55    stash session-specific information.  For more information, please see the
 56    SQLAlchemy docs:
 57    https://docs.sqlalchemy.org/en/stable/orm/session_api.html#sqlalchemy.orm.session.Session.params.info
 58
 59    By placing the active ``request`` in the "info" dict, developers will be
 60    able to access the active Pyramid request from an instance of an SQLAlchemy
 61    object in one of two ways:
 62
 63    - Classic SQLAlchemy. This uses the ``Session``'s utility class method:
 64
 65      .. code-block:: python
 66
 67          from sqlalchemy.orm.session import Session as sa_Session
 68
 69          dbsession = sa_Session.object_session(dbObject)
 70          request = dbsession.info["request"]
 71
 72    - Modern SQLAlchemy. This uses the "Runtime Inspection API":
 73
 74      .. code-block:: python
 75
 76          from sqlalchemy import inspect as sa_inspect
 77
 78          dbsession = sa_inspect(dbObject).session
 79          request = dbsession.info["request"]
 80    """
 81    dbsession = session_factory(info={"request": request})
 82    zope.sqlalchemy.register(
 83        dbsession, transaction_manager=transaction_manager
 84    )
 85    return dbsession
 86
 87
 88def includeme(config):
 89    """
 90    Initialize the model for a Pyramid app.
 91
 92    Activate this setup using ``config.include('tutorial.models')``.
 93
 94    """
 95    settings = config.get_settings()
 96    settings['tm.manager_hook'] = 'pyramid_tm.explicit_manager'
 97
 98    # Use ``pyramid_tm`` to hook the transaction lifecycle to the request.
 99    # Note: the packages ``pyramid_tm`` and ``transaction`` work together to
100    # automatically close the active database session after every request.
101    # If your project migrates away from ``pyramid_tm``, you may need to use a
102    # Pyramid callback function to close the database session after each
103    # request.
104    config.include('pyramid_tm')
105
106    # use pyramid_retry to retry a request when transient exceptions occur
107    config.include('pyramid_retry')
108
109    # hook to share the dbengine fixture in testing
110    dbengine = settings.get('dbengine')
111    if not dbengine:
112        dbengine = get_engine(settings)
113
114    session_factory = get_session_factory(dbengine)
115    config.registry['dbsession_factory'] = session_factory
116
117    # make request.dbsession available for use in Pyramid
118    def dbsession(request):
119        # hook to share the dbsession fixture in testing
120        dbsession = request.environ.get('app.dbsession')
121        if dbsession is None:
122            # request.tm is the transaction manager used by pyramid_tm
123            dbsession = get_tm_session(
124                session_factory, request.tm, request=request
125            )
126        return dbsession
127
128    config.add_request_method(dbsession, reify=True)

Here we align our imports with the names of the models, Page and User.

Migrate the database with Alembic

Now that we have written our models, we need to modify the database schema to reflect the changes to our code. Let's generate a new revision, then upgrade the database to the latest revision (head).

On Unix:

$VENV/bin/alembic -c development.ini revision --autogenerate \
    -m "use new models Page and User"
$VENV/bin/alembic -c development.ini upgrade head

On Windows:

%VENV%\Scripts\alembic -c development.ini revision \
    --autogenerate -m "use new models Page and User"
%VENV%\Scripts\alembic -c development.ini upgrade head

Success executing these commands will generate output similar to the following.

2021-01-07 08:00:14,550 INFO  [alembic.runtime.migration:155][MainThread] Context impl SQLiteImpl.
2021-01-07 08:00:14,551 INFO  [alembic.runtime.migration:158][MainThread] Will assume non-transactional DDL.
2021-01-07 08:00:14,553 INFO  [alembic.autogenerate.compare:134][MainThread] Detected added table 'users'
2021-01-07 08:00:14,553 INFO  [alembic.autogenerate.compare:134][MainThread] Detected added table 'pages'
2021-01-07 08:00:14,558 INFO  [alembic.autogenerate.compare:622][MainThread] Detected removed index 'my_index' on 'models'
2021-01-07 08:00:14,558 INFO  [alembic.autogenerate.compare:176][MainThread] Detected removed table 'models'
  Generating <somepath>/tutorial/tutorial/alembic/versions/20210107_bc9a3dead43a.py ...  done
2021-01-07 08:00:21,318 INFO  [alembic.runtime.migration:155][MainThread] Context impl SQLiteImpl.
2021-01-07 08:00:21,318 INFO  [alembic.runtime.migration:158][MainThread] Will assume non-transactional DDL.
2021-01-07 08:00:21,320 INFO  [alembic.runtime.migration:517][MainThread] Running upgrade 90658c4a9673 -> bc9a3dead43a, use new models Page and User

Alembic overview

Let's briefly discuss our configuration for Alembic.

In the alchemy cookiecutter's development.ini file, the setting for script_location configures Alembic to look for the migration script in the directory tutorial/alembic. By default Alembic stores the migration files one level deeper in tutorial/alembic/versions. These files are generated by Alembic, then executed when we run upgrade or downgrade migrations. The setting file_template provides the format for each migration's file name. We've configured the file_template setting to make it somewhat easy to find migrations by file name.

At this point in this tutorial, we have two migration files. Examine them to see what Alembic will do when you upgrade or downgrade the database to a specific revision. Notice the revision identifiers and how they relate to one another in a chained sequence.

See also

For further information, see the Alembic documentation.

Edit scripts/initialize_db.py

We haven't looked at the details of this file yet, but within the scripts directory of your tutorial package is a file named initialize_db.py. Code in this file is executed whenever we run the initialize_tutorial_db command, as we did in the installation step of this tutorial.

Note

The command is named initialize_tutorial_db because of the mapping defined in the [console_scripts] entry point of our project's setup.py file.

Since we've changed our model, we need to make changes to our initialize_db.py script. In particular, we'll replace our import of MyModel with those of User and Page. We'll also change the the script to create two User objects (basic and editor) as well as a Page, rather than a MyModel, and add them to our dbsession.

Open tutorial/scripts/initialize_db.py and edit it to look like the following:

 1import argparse
 2import sys
 3
 4from pyramid.paster import bootstrap, setup_logging
 5from sqlalchemy.exc import OperationalError
 6
 7from .. import models
 8
 9
10def setup_models(dbsession):
11    """
12    Add or update models / fixtures in the database.
13
14    """
15    editor = models.User(name='editor', role='editor')
16    editor.set_password('editor')
17    dbsession.add(editor)
18
19    basic = models.User(name='basic', role='basic')
20    basic.set_password('basic')
21    dbsession.add(basic)
22
23    page = models.Page(
24        name='FrontPage',
25        creator=editor,
26        data='This is the front page',
27    )
28    dbsession.add(page)
29
30
31def parse_args(argv):
32    parser = argparse.ArgumentParser()
33    parser.add_argument(
34        'config_uri',
35        help='Configuration file, e.g., development.ini',
36    )
37    return parser.parse_args(argv[1:])
38
39
40def main(argv=sys.argv):
41    args = parse_args(argv)
42    setup_logging(args.config_uri)
43    env = bootstrap(args.config_uri)
44
45    try:
46        with env['request'].tm:
47            dbsession = env['request'].dbsession
48            setup_models(dbsession)
49    except OperationalError:
50        print('''
51Pyramid is having a problem using your SQL database.  The problem
52might be caused by one of the following things:
53
541.  You may need to initialize your database tables with `alembic`.
55    Check your README.txt for description and try to run it.
56
572.  Your database server may not be running.  Check that the
58    database server referred to by the "sqlalchemy.url" setting in
59    your "development.ini" file is running.
60            ''')

Only the highlighted lines need to be changed.

Populating the database

Because our model has changed, and to repopulate the database, we need to rerun the initialize_tutorial_db command to pick up the changes we've made to the initialize_db.py file.

On Unix

$VENV/bin/initialize_tutorial_db development.ini

On Windows

%VENV%\Scripts\initialize_tutorial_db development.ini

There should be no output to your console to indicate success.

View the application in a browser

We can't. At this point, our system is in a "non-runnable" state; we'll need to change view-related files in the next chapter to be able to start the application successfully. If you try to start the application (see Start the application) and visit http://localhost:6543, you'll wind up with a Python traceback on your console that ends with this exception:

AttributeError: module 'tutorial.models' has no attribute 'MyModel'

This will also happen if you attempt to run the tests.