Diving Into Archetypes

A whole bunch of great examples are available for Archetypes, so rather than constructing one for the book, I show ArchExample, which comes with the Archetypes installation. This adds a content type called article to give a rough example of the power of Archetypes.

Article.py contains the main product code. You'll see that the code looks quite different from previous examples. It contains schemas

StringField("blurb",
            searchable = 1,
            widget = TextAreaWidget(),
            ),

This piece of code denotes that you have an attribute on the content type called blurb, that it's a string, and that it's going to be shown as an HTML text area. I discuss all the options for fields and widgets in a moment. For now, take a look at the content type in Plone. In Figure 13-1, I've added the blurb content type.

Figure 13-1. The blurb part of the content type

With just those four lines of code, you've added a field to your content type. The standard form elements that you'd expect in Plone are present; if you edit something and come back later, the old value is shown, errors are handled gracefully, and so on. As a demonstration of how easy it is to modify, change the label in the form to appear as Article Blurb, and change the field to be required. To do this, make the following changes:

StringField("blurb",
            required = 1,
            searchable = 1,
            widget = TextAreaWidget(label="Article Blurb"
            ),

Here you've added the required = 1 parameter, which makes this field required, and a label parameter. If you restart Plone and add a new article, the user interface has now changed to reflect the new schema. The field is now called Article Blurb

Figure 13-2. The Article Blurb part of the content type

This isn't just a cosmetic change; the change reflects the alteration of the underlying schema, and this is the real power of Archetypes. Just comparing this to writing products in Python, you can see that all the drudgery has been removed from writing a product. It's like this: If you can define a schema, you can throw together an archetype in absolutely no time. Once you've done that, you can edit it easily and have the changes take effect.

If you do make any of these changes to try the following examples, you'll have to restart Plone. Only then will all the new changes be loaded and properly registered. For more information on this, see the 'Developing with Archetypes” section later in this chapter.

Explaining Schemas, Fields, and Widgets

Merriam-Webster's dictionary definition of a schema is as follows:

schema:

In Archetypes terms, a schema

Each field has one widget defined. A widget

Figure 13-3 shows the relationship between schemas, fields, and widgets.

Figure 13-3. The relationship between schemas, fields, and widgets

Schema((
    StringField('group',
        vocabulary=ARTICLE_GROUPS,
        widget=SelectionWidget(),
        ),
        # other fields here
        )

schema = BaseSchema +  Schema((
    StringField('group',
                vocabulary=ARTICLE_GROUPS,
                widget=SelectionWidget(),
                ),
    ...

from Products.Archetypes.public import BooleanField

from Products.Archetypes.public import IntegerField
# a simple field for age
age = IntegerField('age')

from Products.Archetypes.public import BooleanWidget

from Products.Archetypes.public import IntegerField
from Products.Archetypes.public import IntegerWidget
# a simple field for age
age = IntegerField('age',
       widget=IntegerWidget(label="Your age")
       )

bankAccountNumber = StringField('bank',
   widget=StringWidget(
           label="Bank account number",
           size=20)
         )

Some Example Field and Widget Combinations

This section contains some useful combinations that seem to be commonly used and that you may find as useful examples. For this example, make a drop-down list of your favorite fruits. You'll define the vocabulary attribute as a list of strings. Each of the values in the list is a string of the fruit type; hence, the field type is a StringField. Because you're defining the widget as a SelectionWidget, it'll show up as a drop-down list, like so:

StringField('fruit'
    vocabulary = ["Apple", "Orange", "Mano", "Banana"],
    widget = SelectionWidget(label = "Favourite Fruit")
    )

The ImageField is a useful one for creating and maintaining images in a Plone site. To have a nice simple field that users can upload images into, use the following:

ImageField('portrait',
    widget = ImageWidget(label = "My picture"),
    )

The following is quite a complicated content type. Most content types will have one main field that can take data. If you think of the basic document type, you'll notice a body field that you enter and edit. This one body field is the main text of the content type. So for this standard field, you'll have only a few attributes that should be added.

First, you'll want this field to be searchable, so you should set the searchable attribute. Second, you'll want this field to respond to FTP and WebDAV requests, so you must set the primary attribute (you can find more information on this in the sidebar 'The Primary Field: Marshaling and Responding to FTP and WebDAV”). You'll want multiple content types to be uploaded, so for this reason you set some allowable_content_types. Then, of course, you need to know how to show the field, so for that you set the default_output_type. This then gives you the following field:

TextField('body'
          searchable = 1,
          primary = 1,
          default_output_types = 'text/html',
          allowable_content_types = ('text/plain',
                                     'text/structured',
                                     'text/restructured',
                                     'text/html'),
          widget = RichWidget(label = 'Body'),
          )

The Primary Field: Marhsaling and Responding to FTP and WebDAV

Plone is an object-oriented system where an object has many attributes and can't simply be represented as a plain file. Unfortunately, most the existing protocols such as FTP and WebDAV treat content in exactly this manner. So there needs to be some way to translate between the two, and the primary field does this. By setting the primary field on an object, this field will become the one that's sent and received by these protocols—rather than the entire object.

This is an imperfect solution to a tricky problem, of course. If you've used FTP or External Editor on a file, you'll know that it tries to solve this by placing a number of lines at the top of the page containing key/value pairs for metadata on the object. This is another attempt to solve the same problem.

To set up primary field marshaling, you need to add the following to your schema: marshall=some_marshaller(). Currently, there are only two marshalers: a PrimaryFieldMarshaller, which takes the whole content and places into your object, and an RFC822Marshaller. This second marshaler handles the content for field name/value pairs as used in e-mail. For this chapter's purposes, I'll use the PrimaryFieldMarshaller to handle content with External Editor.

Validations of Input

Although the forms handle the content and some basic errors, such as omitted content, quite well, you'll probably want some more sophisticated error handling. You can make a series of validations to test that the content in your content type is correct. For example, if you have an integer field, you'll probably want to test that the data they've added is what it's meant to be.

To do this, you can add a validation parameter to the field. To test that your IntegerField really is an integer, for example, you'd do this:

from Products.Archetypes.public import IntegerField
from Products.Archetypes.public import IntegerWidget
# a simple field for age
age = IntegerField('age',
      validators=("isInt"),
      widget=IntegerWidget(label="Your age")
      )

Where did isInt come from? Well, isInt is the name of a validator registered in the validation framework. Only a few are in there, but they're quite useful; Table 13-5 describes them. For the exact details of some of these, I recommend reading the code and looking at the regular expressions—you can find these in the validation/validators/__init__.py module in your Products directory:

Table 13-5. Available Validations

You can also register your own validations. A validator is actually a simple class that implements the ivalidator interface. Two are already done: a RegexValidator that verifies a regular expression and a RangeValidator that verifies a range of values. To register a new validator that checks that a user is between, say, 0 and 150 years old (seems reasonable), you'd add the following to your content type in the Python module, before creating the field:

from validation.validators.validator import RangeValidator
from validation import validation
 
# the RangeValidator takes a name for the validator
# and a start and end value
validAge = RangeValidator("validAge", 0, 150)
validation.register(validAge)

Then you can change your validator to the following:

validators=("isInt","validAge"),

First, the code will check that you have a valid integer; second, it'll check that the integer is within the range of sensible ages. If you wanted to add a totally new validator that did something other than a regular expression or range validation, you'd need to add a new validation system. For this example, make a validation that a date is between two values. In the following example, this is called DateRangeValidator and will return a boolean value if the given date fits between the two given dates. This could be useful to assert that a vacation is within the school holidays.

So, now define a new validator called DateRangeValidator in the validators module. This will allow you to register date ranges to assert that the given date falls in the middle. To do this, you'll use the Zope DateTime object (which is covered in more detail in Appendix A). A validator is simple—it's a class that has a name and will respond to the __call__ method by checking the date. The following is the DateRangeValidator class, which was added to the validators module:

from DateTime import DateTime
 
class DateRangeValidator:
    __implements__ = (ivalidator,)
 
    def __init__(self, name):
        self.name = name
 
    def __call__(self, value, *args, **kwargs):
        min, max = args[:2]
        if not isinstance(value, DateTime):
            value = DateTime(value)
 
    return min < value and value < max

After restarting Zope, you can now register a new validation, like so:

from validation.validators.validator import DateRangeValidator
from validation import validation
from DateTime import DateTime
 
christmas = DateRangeValidator("ChristmasHolidays",
    DateTime('12/18/2004'),
    DateTime('01/09/2005'),)
validation.register(christmas)

Then you can create a validator in your schema in the following manner:

validators=("christmas",)

Overriding Views and Actions in the Base Class

Archetypes creates default views and actions for you based on a standard set of requirements that will suit many needs. The actions are view, edit, and properties, of course; references is another action you'll look at it a moment. You won't find any view or edit page templates for the object; those are generated automatically for you by Archetypes. However, you can override them.

I expect that in most cases you'll want to override the view method and provide one of your own; the default is basic and isn't aimed at being a perfect page (of course, being a Plone page, it's better than your average content management system). However, you may have specific needs that relate to the content; perhaps it's a matter of presenting your content in a certain way.

Archetypes actually creates a class for each content type. Much in the same way that you created a class for the source code type in the previous chapter, you can create a base class for your archetype. This base class is called BaseContent and is available in the public module of Archetypes for importing. This BaseContent class defines all the things that Archetypes needs to know. Making this class provides you with an opportunity to override almost anything you want in the class.

As I now show, there are two parts to this. First, you make an action that's to be used by the factory type information; in Archetypes you do this by assigning the actions attribute of the class. For example:

from Products.Archetypes.public import BaseContent
 
class Article(BaseContent):
    # other stuff
    actions = ({ 'id': 'view',
                 'name': 'View',
                 'action': 'string:${object_url}/article_view',
                 'permissions': (CMFCorePermissions.View,)
                },)

Second, you need to create a page template for the actual view of the object called article_view. This string defines a page template that the content type will locate in the skin for this product. In this case, you'll find the matching page template on the file system in the skins/archexample directory of ArchExample. You can also find a copy of this page template in Appendix B. This can be as simple or as complicated a page template as you'd like.

You have to restart Plone for the changes to take effect. In this case, you're changing an action that's installed into the portal_types tool whenever the product is installed. For this reason, if you do change this action, you'll need to reinstall the product. To do this in the Plone interface, click plone setup

All the elements in the factory type information can be overridden by creating an attribute of that name on the object. To override the content_icon setting, you'd make an attribute called content_item. For example:

class SomeProduct(BaseContent):
          """ Some product """
          content_icon = "some_icon.gif"

Overriding the Default Methods

In Table 13-2, I mentioned the option of overriding some of the default methods that occur on content types, such as accessor and mutator. This is an advanced option that allows you to manipulate the way fields are edited.

Just as with the source code project in the last chapter, you access these attributes or fields using accessors and mutator methods. Some default methods are available for you. If your field name is blurb, then these methods will be getBlurb and setBlurb. The term get

However, you may want to do something different in the accessor or mutator. Suppose you wanted to filter the value of a field, such as always correcting the spelling of your company name or changing the value of some other fields when a certain field changes. Then you could do so by overriding the default methods. In the following example, you'll make a new method called getSpecialBlurb that takes the blurb someone has entered and manipulates it before returning to the client. In this case, you're replacing the text Perl

getSpecialBlurb Article

class Article(BaseContent):
 
    def getSpecialBlurb(self):
        """ The view for an article """
        blurb = self.getField('blurb').get(self)
        blurb = blurb.replace('Perl', 'Python')
        return blurb

You'll also need to change your field so that it uses this method:

StringField('blurb',
    searchable=1,
    widget=TextAreaWidget(),
    accessor="getSpecialBlurb",
),

In this example, anytime the blurb field is accessed in a view or edit page, for example, the value of getSpecialBlurb is returned. Archetypes knows to access that method because the name of the method is defined as a string value passed to the accessor parameter. There's one bit of trickery involved—to access the raw value of the attribute, you need to get the field and then call the get method. This is the rather confusing blurb = self.getField('blurb').get(self) line. The pattern of getting the field and then calling a method on it is actually quite common in Archetypes.

The practical upshot of this is that now, no matter when or how a person types in the word Perl

Using Unique IDs

The concept of unique IDs is simple but something that was missing in Zope. Originally Zope developers assumed using the path to an object would do; unfortunately, this has shown to be inadequate. As a simple example, consider what happens when someone moves the location of a document—your unique reference to that document has been lost. A unique ID is a truly useful tool to have. Archetypes creates a unique ID on every object it creates and saves it for you on the object. It also saves it in a separate catalog for you called uid_catalog.

You can see the uid_catalog in the ZMI, and it's like the portal_catalog object except it's missing a little bit of extra Application Programming Interface (API) wrapping and all the indexes that the former contains. It's pretty simple to get an object now by going through this catalog. All you have to do is search the catalog for your object. For example, the following Script (Python) object can pull any registered object, provided you know its UID:

##paramaters=objectId
results = context.uid_catalog(UID=objectId)
return results[0].getObject()

But where this really is useful is in references between existing objects. Say you want to reference one or more separate images with your article. These images could be images uploaded by other users at another date, perhaps part of an image database. If those images are Archetypes objects, then you could add a field to your article schema that reads as follows:

ReferenceField("images"
    allowed_types=("Archetype Images",),
    multivalued=1,
    ),

The user will now get a drop-down box of all the Archetype Image objects and be able to choose them. Behind the scenes it's making a reference of that object's UID through the catalog. You'll find an excellent example on references in a product called ACME in the Archetypes CVS repository at http://sf.net/projects/archetypes.

Altering Widgets

A commonly asked question is, why does this widget do this? Another is, why does this widget look like this? Often the answer is something like, Because it does—now go and write your own. Since widgets are what the client sees, these requirements are often client driven.

All the widgets are represented as page template macros on the file system. So it's pretty simple to alter widgets or make your own. If you click portal_skins

portal_skins Actually, the term macro

The view macro displays on the view page and is a read-only, user-friendly view of the item. The edit macro is the one shown on the edit page, and it's the macro where a user edits the data. The search macro is called when you're assembling search pages for the code; it sometimes looks like the edit macro, although it may not. A string field may be edited as a string field and viewed as a string, but it's searched using a drop-down box to select from all the options available.

So, in the example product, assume you have a string field that's a person's e-mail address, and you want to show this is a clickable e-mail link. For this you'll make a new macro. In this case, you don't need to actually override the edit and search macros—it's just a string after all. So, make a new a page template called email_widget.pt, as follows, which you'll place in the skin of your product:

<html xmlns:tal="http://xml.zope.org/namespaces/tal"
      xmlns:metal="http://xml.zope.org/namespaces/metal"
      i18n:domain="plone">
 
  <body>
    <div metal:define-macro="edit">
      <div metal:use-macro="here/widgets/string/macros/edit" />
    </div>
 
    <div metal:define-macro="search">
      <div metal:use-macro="here/widgets/string/macros/search" />
    </div>

For the view, you need to make the string show as a mailto link, which is just the following simple adjustment:

    <div class="field" metal:define-macro="view">
        <a href="#" tal:attributes="href string:mailto:${accessor}"
          tal:content="accessor">email</a>
 
    </div>
</body>
</html>

Now that you've defined a page template that contains your code, you can simply reference the name of the template as the macro in your widget. In the following code, you define the e-mail field and a StringWidget as normal. You then change your macro to use your nice new email_template macro, like so:

StringField('email',
    validators = ('isEmail',),
    widget = StringWidget(
        label='Email',
        macro='email_template'
    )
)

At this point you're just changing the macro of an existing widget. Making an entirely new widget is just a matter of defining a new widget and then registering it. All widgets share the same base class. The following is a new module called EmailWidget.py that's placed inside the ArchExample directory. It creates a new widget and then registers it into the registry. Note that the macro property of the widget (highlighted) is set to the value of the template:

from Products.Archetypes.Widget import TypesWidget
from Products.Archetypes.Registry import registerWidget
 
class EmailWidget(TypesWidget):
    _properties = TypesWidget._properties.copy()
    _properties.update({
       'macro' : "email_template",
       'size' : '30',
       'maxlength' : '255',
       })
 
registerWidget(EmailWidget,
    title='String',
    description='Renders a clickable email field',
    used_for=('Products.Archetypes.Field.StringField',)
)

To include this in your article, you can now directly import the EmailWidget and use without having to explicitly define the macro, like so:

from EmailWidget import EmailWidget
 
StringField('email',
    validators = ('isEmail',),
    widget = EmailWidget(
        label='Email',
    )
)

Developing Folderish Objects

You've already worked with folderish objects a lot in Plone, but you may not have known it. A folderish object is one that exhibits similar characteristics to a folder or directory, namely, that it can contain separate individual objects. There really is nothing special about a folderish object—just that it inherits from a certain base class to provide its properties, and new content can be added to it.

Folderish objects are useful for you to make for several reasons. They provide a simple way to create collections of disparate objects in one location. They also allow users to manage the contents using the standard Plone user interface, without you having to do anything. Generally, it's best to keep the folder quite simple, and the logic should be on your objects and in the objects' workflow. However, some exceptions to that always exist, and a classic example is the family of collectors, CMFCollector and PloneCollector both of which are quite complicated collector objects that provide logic for a bunch of collector issues.

The easiest way again to make a folderish object is to use Archetypes. You've seen that there's a BaseContent type that handles all the work needed to create a nonfolderish object. Well, there's also a BaseFolder content type that contains everything for creating a folder. Futher, you'll find a special schema for folders, since folderish object also usually have a description. To make a folderish type, just ensure that you change your base classes and your schema. For example, possibly the simplest folder is as follows:

from Products.Archetypes.public import BaseFolder, BaseFolderSchema
 
schema = BaseFolderSchema
 
class SimpleFolder(BaseFolder):
    """A simple folderish archetype"""
    schema = schema
 
registerType(SimpleFolder)

If you're going to store a huge amount of content in a folder, then a folder tends to become inefficient at about 100 objects. That's a rather arbitrary figure that should be taken with a pinch of salt; normal folderish objects are designed to be fast for small numbers of object. If you wanted, you could use a binary tree folder, which stores the objects internally in a more efficient binary tree, rather than a Python dictionary. To use this, just import BaseBTreeFolder and BaseBTreeFolderSchema and use these in your object instead. As far as developing a product is concerned, they work just the same, but unless you put a lot of data into them, you wouldn't likely notice any difference. I've stored more than 100,000 objects in BTreeFolder, and it's always responded well.

Handling Microsoft Office Content

Handling Microsoft Office content such as Word and Excel documents is a bane that all content management systems face at some point. However, this is really the case for every type of content—Microsoft Office, OpenOffice.org, Portable Document Format (PDF), images, and so on. Using these content forms on Web sites usually causes a few problems; this is well known, of course. Editing is clumsy because clicking a document causes you to download it or, even worse, open it in your browser. When you've finished, you have to upload back up to the Web site, which can normally mean clicking around a bit to get back to it. When the content is online, the contents aren't searchable because they aren't in a plain-text format the catalog can understand. Further, you can't view the content online because again it isn't in a Web-friendly format.

You have a couple of solutions for solving the first problem of editing content. If again you assume most of your users are using Windows, then using WebDAV can be tricky for you because Microsoft's Web folder implementation is of questionable quality. Instead, External Editor fulfills this function quite well—if Plone can tell External Editor that this is a Word file, it'll open in Word.

As for the rest, well, Archetypes has a built-in transformation package, called Portal Transforms, that handles the transformation of content types. This can take a file in a certain format and transform it into HTML, which will then be cataloged, and the HTML will display to the user in the user interface. It does this by using an external transformation process to transform the data and read the results. For example, if you're using Windows (on the server), then Portal Transforms will take the uploaded Word document and start a Component Object Model (COM) object that transforms it.

This all happens behind the scenes for you; all you have to worry about is ensuring that any requirements for your transforms are installed and operational. You can do lots of different types of transforms, and there's more than one way to transform the data.

The OpenOffice.org suite provides excellent transformation of Microsoft content, so if you're running on a non-Microsoft platform, this is an excellent way to transform that content. I've also used wvWare (http://wvware.sourceforge.net/) quite successfully as another way of transforming content. All these options are available to you; however, they aren't trivial to set up. Once they're set up, of course, people usually report getting higher-quality transformations than when using a Microsoft server.

I recommend thinking first about exactly what you want to transform and then taking a look at the Portal Transforms source code to see if you can find a transformation that will do the job for you.

Going Further: Writing Content Types in a UML Diagram

So you have a complicated content type and the idea of writing it manually is boring? Well, never fear, because ArchGenXML is here! This is a neat product that allows you to write your content in a UML graphical modeling tool and develop a product straight from that.

Phillip Auspberg developed ArchGenXML, and Jens Klein has provided some excellent documention. With kind permission, I'll reproduce some of those diagrams here.

The UML modeling tool must support a standard Extensible Markup Language (XML) generation system called XML Metadata Interchange (XMI). This tool has been tested with the following programs:

• ObjectDomain (commercial, with a free demo for 30 classes) from http://www.objectdomain.com
• ArgoUML (free) from http://argouml.tigris.org
• Poseidon (commercial, based on ArgoUML) from http://www.gentleware.com
• Sybase Powerdesigner (commercial, demo download) from http://www.sybase.com

There are slightly different export properties for each of these, but if you take a look in the ArchGenXML samples, you'll see example outputs from the UML tools. Figure 13-6 shows a class project that shows a few objects: room, person project, resource, and task. There are relationships between all these objects, and there's even Python code specified on some of the objects.

Figure 13-6. A complex ArchGenXML example

The quickest way to test this is to go into the ArchGenXML folder on your file system that you kept around from the original install and then change into the examples folder. In that folder you'll see the mkdemo script. To run it, just type ./mkdemo

python ../ArchGenXML.py outline.zargo outline_argo

You then need to copy the product folder created (for example, outline_argo) into your Products directory and restart your Zope instance. If you take a peek inside the outline_argo directory, you'll see that everything has been created for you—the entire directory and all the relevant product initialization. Click plone setup, select Add/Remove Products, and install the outline_argo product. You can now add an outline object that has all the schema defined in the UML.

That's about as rapid development as you can get, and there are many, many options inside ArchGenXML for defining abstract classes, code fragments, and so on. Furthermore, generating code this way easily allows you to make changes and have great documentation. The only real drawback is a lack of round-tripping

http://plone.org/documentation/archetypes/ArchetypesDeveloperGuide/index_html#archgenxml-generating-archetypes-using-uml

Storing Your Content in a SQL Database

For almost all of this book, the content you've seen has been stored in the Zope object database. I've shown you how to store content on the file system, but a commonly asked question is, How do you store it in a relational database? Placing it inside a relational database is often a good thing to do if the following are true:

• Your structure is rigid, and the schema doesn't change often (although Archetypes mitigates this aspect).
• You have other applications that can or do access the relational database.
• There are tools or other requirements that are already fulfilled by a relational database.
• You have a large amount of repetitive data that's updated often.

In more a traditional CGI environment, you'd probably write some SQL statements for pulling the content out of a relational database. You can do this if you really want to by using ZSQL methods. These are useful for many situations and are covered in great detail in the Zope Book and several other books, but they can't be used to directly store content. A ZSQL method stores SQL statements and makes queries into your relational database based upon those queries. This is great if you want to make simple queries into relational database, but it doesn't persist Python classes or content types, and it really isn't the Plone way of thinking. You'll find an excellent chapter in the Zope Book on this at http://zope.org/Documentation/Books/ZopeBook/2_6Edition/RelationalDatabases.stx.

SQLStorage uses the fact that Archetypes provides an extra layer on top of normal objects to allow for special persistence mechanisms. Zope provides many different relational database adapters, and SQLStorage plugs into those. Currently there are two adapters, Postgres and MySQL. However, if the database supports standard SQL, then there's no reason one of these existing SQLStorage formats can't be used. If you find that your database adapter needs to do something different, then it wouldn't be difficult to alter the SQL statements inside SQLStorage so it worked with another database.