# Introduction Marrow Mongo is a collection of small, focused utilities written to enhance use of the [PyMongo native MongoDB driver](http://api.mongodb.com/python/current/) without the overhead, glacial update cycle, complexity, and head-space requirements of stateful *active mapper* patterns. This project grew out of the need (both personal and commercial) to find a viable, simple, well-tested alternative to existing ODMs. We believe that Marrow Mongo hits the Goldilocks zone for a supportive MongoDB experience in Python without getting in the way, offering elegant and Pythonic approaches to document storage modelling, access, and interaction. This is a living document, evolving as the framework evolves. You can always [browse any point in time](https://github.com/marrow/mongo/commits/book) within the [source repository](https://github.com/marrow/mongo/tree/book) to review previous versions of these instructions. (Try using the "edit this page" link in the upper right if viewing this document on the [official site](https://mongo.webcore.io/).) ## Overview [![Latest version.](https://img.shields.io/pypi/v/marrow.mongo.svg?style=flat)](https://pypi.python.org/pypi/marrow.mongo) [![Latest tag.](https://img.shields.io/github/tag/marrow/mongo.svg)](https://github.com/marrow/mongo/releases/latest)\ [![Subscribe to project activity on Github.](https://img.shields.io/github/watchers/marrow/mongo.svg?style=social\&label=Watch)](https://github.com/marrow/mongo/subscription) [![Star this project on Github.](https://img.shields.io/github/stars/marrow/mongo.svg?style=social\&label=Star)](https://github.com/marrow/mongo/subscription) [![Fork this project on Github.](https://img.shields.io/github/forks/marrow/mongo.svg?style=social\&label=Fork)](https://github.com/marrow/mongo/fork) ### Plugin Package Namespaces Explicit is better than implicit, with fields, traits, and document classes registered as `entry_point` plugins and made accessible through the standard import mechanism. [Learn more about plugin registration and discovery.](https://github.com/marrow/mongo/tree/c047b93da447084919dd02c068a5548c59d1c8fe/guide/plugins.md) ```python from marrow.mongo import Document, Index from marrow.mongo.field import String from marrow.mongo.trait import Queryable ``` ### Declarative document modeling. Instantiate field objects and associate them with custom `Document` sub-classes to model your data declaratively. [Learn more about constructing documents.](https://github.com/marrow/mongo/tree/c047b93da447084919dd02c068a5548c59d1c8fe/guide/documents.md) ```python class Television(Document): model = String() ``` ### Refined, Pythonic *data access object* interactions. Utilize `Document` instances as attribute access mutable mappings with value typecasting, directly usable with PyMongo APIs. Attention is paid to matching Python language expectations, such as allowing instantiation using positional arguments. Values are always stored in the PyMongo-preferred MongoDB native format, and cast on attribute access as needed. [Learn more about interacting with documents.](https://github.com/marrow/mongo/tree/c047b93da447084919dd02c068a5548c59d1c8fe/guide/instances.md) ```python tv = Television('D50u-D1') assert tv.model == \ tv[~Television.model] == \ tv['model'] == \ 'D50u-D1' ``` ### Collection and index metadata, and creation shortcuts. Keep information about your data model with your data model and standardize access. [Learn more about indexing.](/guide/indexes) ```python class Television(Queryable): __collection__ = 'tv' model = String() brand = String() _model = Index('model') collection = Television.create_collection(database) Television('D50u-D1').insert_one() ``` ### Filter construction through rich comparison. Construct filter documents through comparison of (or method calls on) field instances accessed as class attributes. [Learn more about querying documents.](https://github.com/marrow/mongo/tree/c047b93da447084919dd02c068a5548c59d1c8fe/guide/querying.md) ```python exact = Television.model == 'D50u-D1' prefix = Television.model.re(r'^D50\w') tv_a = Television.find_one(exact) tv_b = Television.find_one(prefix) assert tv_a.model == tv_b.model == 'D50u-D1' assert tv_a['_id'] == tv_b['_id'] ``` ### Parametric filter, projection, sort, and update document construction. Many Python *active record* object relational mappers (ORMs) and object document mappers (ODMs) provide a short-hand involving the transformation of named parameters into database concepts. [Learn more about the parametric helpers.](https://github.com/marrow/mongo/tree/c047b93da447084919dd02c068a5548c59d1c8fe/guide/parametric.md) ```python filter_doc = F(Television, model__ne='XY-zzy') update_doc = U(Television, set__brand='Vizio') tv = Television.find_one(model='D50u-D1') assert tv.brand == 'Vizio' ``` ### Advanced GeoJSON support. Marrow Mongo comes with [GeoJSON](http://geojson.org) batteries included, having extensive support for querying, constructing, and manipulating GeoJSON data. [Learn more about working with geospatial data.](https://github.com/marrow/mongo/tree/c047b93da447084919dd02c068a5548c59d1c8fe/guide/geospatial.md) ```python position = Point(longitude, latitude) collection.find(Battleship.location.near(position)) ``` ## Code Quality [![Release build status.](https://img.shields.io/travis/marrow/mongo/master.svg?style=flat)](https://travis-ci.org/marrow/mongo/branches) [![Release test coverage.](https://img.shields.io/codecov/c/github/marrow/mongo/master.svg?style=flat)](https://codecov.io/github/marrow/mongo?branch=master) [![Release code health.](https://landscape.io/github/marrow/mongo/master/landscape.svg?style=flat)](https://landscape.io/github/marrow/mongo/master) [![Status of release dependencies.](https://img.shields.io/requires/github/marrow/mongo.svg)](https://requires.io/github/marrow/mongo/requirements/?branch=master) ### Guaranteed to be fully tested before any release. We utilize [Travis](https://travis-ci.org/marrow/mongo/) continuous integration, with test coverage reporting provided by [Codecov.io](https://codecov.io/gh/marrow/mongo/). We also monitor requirements for security concerns and deprecation using [Requires.io](https://requires.io/github/marrow/mongo/requirements/?branch=master). Extensive static analysis through [Landscape.io](https://landscape.io/marrow/mongo/), proactive use of tools such as [pre-commit](http://pre-commit.com) with [plugins](https://github.com/marrow/mongo/blob/develop/.pre-commit-config.yaml) such as the infosec analyzer [OpenStack Bandit](https://wiki.openstack.org/wiki/Security/Projects/Bandit), and various linting tools help to keep code maintainable and secure. ### Extensively documented, with a > 1:1 code to comment ratio. Every developer has run into those objects that fail to produce sensible or useful programmers' representation, generate meaningless exception messages, or fail to provide introspective help. With more documentation in the code than code, you won't find that problem here. Code should be self-descriptive and obvious; we feel comments and *docstrings* are integral to this. ### A considered road map. Changes to the library demand [meditation](https://github.com/marrow/mongo/projects) to ensure feature creep and organic growth are kept in check. Where possible, solutions involving objects passed to standard PyMongo functions and methods are preferred to solutions involving wrapping, proxying, or middleware. All but minor changes are isolated in [pull requests](https://github.com/marrow/mongo/pulls) to aid in code review. ## MIT Licensed The [MIT License](/end-matter/license) is highly permissive, allowing **commercial** and **non-commercial** *use*, *reproduction*, *modification*, *republication*, *redistribtution*, *sublicensing*, and *sale* of the software (and associated documentation) or its components. The license notice must be included in the reproduced work, and any warranty or liability on behalf of the [Marrow Open Source Collective](https://github.com/marrow/) or [project contributors](https://github.com/marrow/mongo/graphs/contributors) waived. You are effectively free to deal in this software however you choose, **without commercial hinderance**. ## Code Metrics | `marrow.mongo` as of `a889491` | Value | | ------------------------------ | ------ | | **Total Lines** | 2,976 | | **SLoC** | 1,479 | | **Logical Lines** | 840 | | **Tests** | 305 | | **Functions** | 57 | | **Classes** | 41 | | **Modules** | 23 | | **Average Complexity** | 2.5 | | **Complexity 95th %** | 6 | | **Maximum Complexity** | 17 | | **# > 15 Complexity** | 1 | | **Bytecode Size** | 71 KiB | # Installation Installation is easy using the `pip` package manager. **Requirements** * [Python](https://www.python.org) 2.7 or 3.2 and above, or compatible runtime such as [Pypy](http://pypy.org) or Pypy3. * An accessible [MongoDB](https://www.mongodb.com/) installation; some features may require MongoDB version 3.2, decimal support requires version 3.4. ```bash pip install marrow.mongo ``` ## Dependencies > ### info::Dependency Isolation > > We strongly recommend always using a container, virtualization, or sandboxing environment of some kind when developing using Python; installing things system-wide is yucky (for a variety of reasons) nine times out of ten. > > We prefer light-weight [virtualenv](https://virtualenv.pypa.io/en/latest/virtualenv.html), others prefer solutions as robust as [Vagrant](http://www.vagrantup.com). Python dependencies will be automatically installed when `marrow.mongo` is installed: * A modern (3.2 or newer) version of the `pymongo` package. * The `marrow.package` plugin and canonical object loader. * The `marrow.schema` declarative syntax toolkit. If you add `marrow.mongo` to the `install_requires` argument of the call to `setup()` in your application's `setup.py` file, `marrow.mongo` will be automatically installed and made available when your own application or library is installed. We recommend using *less than* version numbers to ensure there are no unintentional side-effects when updating. Use `marrow.mongo<1.2` to get all bugfixes for the current release, and `marrow.mongo<2.0` to get bugfixes and feature updates while ensuring that backwards-incompatible changes are not installed without warning. There are a few conditional, tag-based dependencies. To utilize these optional tags add them, comma separated, beween square braces. This may require shell escaping or quoting. ```bash pip install 'marrow.mongo[scripting,logger]' ``` ## Package Flags * `development` Install additional utilities relating to testing and contribution, including `pytest` and various plugins, static analysis tools, debugger, and enhanced REPL shell. * `scripting` Pulls in the [Javascripthon](https://github.com/azazel75/metapensiero.pj) Python to JavaScript *transpiler* to enable use of native Python function transport to MongoDB. (E.g. for use in map/reduce, stored functions, etc.) * `logger` Logging requires knowledge of the local host's timezone, so this pulls in the `tzlocal` package to retrieve this information. ## Development Version Development takes place on [GitHub](https://github.com/) in the [marrow.mongo](https://github.com/marrow/mongo/) project. Issue tracking, documentation, and downloads are provided there. Installing the current development version requires [Git](http://git-scm.com/), a distributed source code management system. If you have Git you can run the following to download and *link* the development version into your Python runtime. ```bash git clone https://github.com/marrow/mongo.git (cd mongo; python setup.py develop) ``` If you would like to make changes and contribute them back to the project, fork the GitHub project, make your changes, and submit a pull request. For more information see the [Contributing](/contributing) section, and [GitHub's documentation](http://help.github.com/). # Contributing Thank you for considering contributing to this project! We welcome contributions large and small, from documentation to code. Following these guidelines helps communicate that you respect the time of the developers managing and developing this open source project. In return, we will recriprocate that respect in addressing your issue, assessing changes, and helping you finalize your pull requests. There are many ways to contribute, from advocacy through writing tutorials or blog posts, improving or translating the documentation, submitting bug reports and feature requests, or writing code which can be incorporated into future Marrow Mongo releases. ### Table of Contents 1. [Asking Questions](/contributing#asking-questions) 2. [First Steps](/contributing#first-steps) 3. [Guidelines](/contributing#guidelines) 4. [Donations and Patreon Support](/contributing#donations-and-patreon-support) ## Asking Questions The first step in determining how best to tackle a concern is to ask about it. There are a number of avenues available for discussion and support. * **Freenode IRC** Likely the best way to get fast turnaround on inquiries is through the Marrow Open Source Collective community on Freenode IRC. Point your IRC client at the `#webcore` channel on `chat.freenode.net:6667` (SSL; non-SSL port 6667) or, if you are viewing this on a medium which supports it and have an IRC client installed which recognizes `irc://` URLs, just [use this link](irc://chat.freenode.net:6667/%23webcore). If you have general Python questions, or general MongoDB questions, [`##python-friendly`](irc://chat.freenode.net:6667/%23%23python-friendly), [`#python`](irc://chat.freenode.net:6667/%23python), or [`#mongodb`](irc://chat.freenode.net:6667/%23mongodb), as appropraite, may be more active. Stack Overflow is also worth considering. * **Stack Overflow** To help develop a comprehensive knowledgebase of problems and solutions, with good metadata, user incentives, and fancy native apps, you can utilize Stack Overflow to ask your questions. Just remember to tag your question [`marrow.mongo`](http://stackoverflow.com/questions/tagged/marrow.mongo) so that it can be found. * **GitHub Issues** For problems relating to bugs or enhancements witin Marrow Mongo itself, please utilize the [Marrow Mongo Issue Tracker](https://github.com/marrow/mongo/issues) provided by GitHub. If you have identified an issue and have already begun work on a feature branch, just issue a pull request instead of creating a ticket, then creating a pull request. Please do not use the issue tracker for general support questions. IRC or Stack Overflow are better resources for quick questions or longer questions, respectively. ## First Steps * **Triage** If you find an [unreviewed ticket](https://github.com/marrow/mongo/issues?q=is%3Aopen+is%3Aissue+no%3Alabel) reporting a bug, try to reproduce it. If the problem appears valid make a note that you confirmed the bug. * **Help Wanted** Look through the [Help Wanted](https://github.com/marrow/mongo/issues?q=is%3Aopen+label%3A"help+wanted"+sort%3Areactions-%2B1-desc) issues on GitHub. Look for tickets with greater numbers of reactions and comments to help gauge impact. If you feel it's a problem you can help solve, ask to be assigned. * **Write Some Documentation** We like to think that Marrow Mongo's documentation is excellent, but there's always room for improvement. Did you find a typo? Do you feel a certain section is unclear? Let us know by leaving a comment on the web version of this document—hover over a paragraph, use the speech bubble on the right—or feel free to contrbute to the [book](https://github.com/marrow/mongo/tree/book). ## Guidelines * **Start small.** It'll be less overwhelming to tackle a small, focused problem, and easier to get feedback. * **Leave feedback.** See an idea being discussed, or a work-in-progress you feel could use your input? Do not hesitate to comment anywhere at any time. * **Large (especially breaking) changes should be discussed.** [Create issues](https://github.com/marrow/mongo/issues/new) for any major changes and enhancements that you wish to make. Discuss things transparently and get community feedback. * **Be excellent to each-other.** Be welcoming to newcomers and encourage diverse new contributors from all backgrounds. See the [Python Community Code of Conduct](https://www.python.org/psf/codeofconduct/). * **Code quality is a key priority.** We love code. To ensure our code is understandable and maintainable long into the future it is important that changes: 1. Ensure cross-runtime (CPython, Pypy) and cross-version (2.7, 3.x) compatibility. 2. Ensure code coverage and code health do not decrease. 3. Ensure changes are up to date with their parent branch and that they do not conflict. ## Donations and Patreon Support Financial contributions are welcome. To arrange a one-time donation, please contact `GothAlice` on IRC (see details above) and for more ongoing support we have [Patreon](https://www.patreon.com/GothAlice) prepared for you. We have rewards (such as inclusion in the [Patrons team on GitHub](https://github.com/orgs/marrow/teams/patrons) and our eternal gratitude) and goals, so please give it a look; every cent helps pay for infrastructure costs and caffeine. So much caffine. If at any point you wish or need to cease recurring donations, do not feel bad. Don't put yourself out on our account, and thank you! # Introduction Documents are the records of [MongoDB](https://www.mongodb.com/), stored in an efficient binary form called [BSON](http://bsonspec.org/), allowing record manipulation that is cosmetically similar to [JSON](http://json.org/). In Python these are typically represented as [dictionaries](https://docs.python.org/3/library/stdtypes.html#mapping-types-dict), Python's native mapping type. Marrow Mongo provides a [`Document`](https://github.com/marrow/mongo/tree/c047b93da447084919dd02c068a5548c59d1c8fe/guide/reference/document.md) mapping type that is directly compatible with and substitutable anywhere PyMongo uses dictionaries. This package utilizes the [Marrow Schema](https://github.com/marrow/schema#readme) declarative schema toolkit and extends it to encompass MongoDB data storage concerns. Its documentation may assist you in understanding the processes involved in Marrow Mongo. At a fundamental level you define data models by importing classes describing the various components of a collection, such as `Document`, `ObjectId`, or `String`, then compose them into a declarative class model whose attributes describe the data structure, constraints, etc. `Field` types and `Document` mix-in classes (*traits*) meant for general use are registered using Python standard [*entry points*](http://setuptools.readthedocs.io/en/latest/setuptools.html#extensible-applications-and-frameworks) and are directly importable from the `marrow.mongo.field` and `marrow.mongo.trait` package namespaces respectively. Within this guide fields are broadly organized into three categories: * **Simple fields** are fields that hold *scalar values*, variables that can only hold one value at a time. This covers most datatypes you use without thinking: strings, integers, floating point or decimal values, etc. * **Complex fields** cover most of what's left: variables that can contain multiple values. This includes arrays (`list` in Python, `Array` in JavaScript), compound mappings (*embedded documents*), etc. * **Complicated fields** are represented by fields with substantial additional logic associated with them, typically through complex typecasting, or by including wrapping objects containing additional functionality. These are separate as they represent substantial additions to core MongoDB capabilities, possibly with additional external dependencies. # Modelling The `Document` class heirarchy is organized to structure both data and the code manipulating it into clearly defined problems, with composable components focused on the principle of least concern. As such, the base class assumes very little; by itself it is a [`MutableMapping`](https://docs.python.org/3/library/collections.abc.html?highlight=abc#collections.abc.MutableMapping) abstract base class-compatible ordered dictionary proxy or wrapper, usable anywhere a mapping is usable. A notable difference is that the constructor only accepts arguments which have discrete fields associated with them. There is no need for a specialized "dynamic" variant. Similarly, we have the philosophy that all documents are embeddable. Top-level documents in a collection, which expect an identifier, should utilize the specialization—*trait*—`Identified`. Begin by importing various components from the `marrow.mongo` package or one of the namespace packages for fields and traits, respectively. ```python from marrow.mongo import Index from marrow.mongo.field import Array, Number, String, ObjectId from marrow.mongo.trait import Queryable ``` To define a schema construct a `Document` subclass. In this example, one to store information about user accounts. We utilize the `Queryable` subclass of `Document`, containing the majority of stateless *active record* behaviour and provide (via `Identified`) an `id` field. All documents and traits you define must ultimately subclass `Document` in order to utilize the metaclass that makes the declarative mechanisms operate. ```python class Account(Queryable): ``` Initially we populate metadata. The first defines the name of the collection to use when *binding* the class to a database, and is optional; you can bind it to a collection directly if you wish, or use it without binding at all. The second is used to specify a default validation level and generate a validation document (schema and/or constraints). ```python __collection__ = 'accounts' __validate__ = 'strict' ``` Populate the class with a few different types of field by assigning `Field` instances as class attributes. Most accounts represent people, who have names. A simple string, with no constraints or transformation options given. Because no default value was given, any attempt to retrieve this attribute on an instance of `Account` prior to assigning one will raise an `AttributeError`, as a value for the field would not exist at all. ```python name = String() ``` Fields missing from the document might be A-OK in some circumstances, but not all. We can mark our acount's `username` as required, resulting in the addition of a constraint when generating the validation document. ```python username = String(required=True) ``` When utilizing default values you may choose to have the default value written immediately into the document. By utilizing the `assign` option the default value will be assigned to the instance immediately upon instantiation, unless passed to the constructor, resulting in the default value being present in the database. This armors records against potential future changes in the default value, if you do not wish such changes to propagate. ```python locale = String(default='en-CA', assign=True) ``` We technically allow storage of any numeric value, either integer or floating point, for our user's age. To prevent explosions if an age is not given we define a default, though this default will not waste storage space in the database by being assigned. ```python age = Number(default=None) ``` Now we define an array of free-form strings to utilize as tags. This is a complex field whose first argument is the type of value it contains and defaults to an empty version of the complex type it represents if assignment is enabled to eliminate the need for boilerplate code. ```python tag = Array(String(), assign=True) ``` Even though `Account` inherits `Identified`, we don't want to gum up construction of new instances by allowing the ID to be defined positionally, so we adapt it. Adapted and redefined fields maintain their original order/position. ```python id = Queryable.id.adapt(positional=False) ``` Lastly we define a unique index on the username to speed up any queries involving that field, and to enforce uniqueness. Because MongoDB's index capabilities are quite expressive, we do not define index features on fields themselves. It is generally a good idea to underscore-prefix non-field attributes. This helps keep fields distinct from non-fields in a visual way and implies they are "protected" or "private" as is customary in Python, though not enforced. ```python _username = Index('username', unique=True) ``` Now that we have a document defined we can move on to exploring how to interact with them. # Management `Document` subclasses utilizing the `Collection` trait (which `Queryable` inherits) gain class-level *active record* behaviours. Additionally, `Collection` inherits `Identified` as well, providing an automatically generated ObjectId field named `id` which maps to the stored `_id` key. There is a fairly substantial number of [collection metadata and calculated properties](https://github.com/marrow/mongo/tree/c047b93da447084919dd02c068a5548c59d1c8fe/guide/reference/trait/collection.md#metadata) available. Before much can be done, it will be necessary to get a reference to a MongoDB connection or database object. Begin by importing the client object from the `pymongo` package. ```python from pymongo import MongoClient ``` Then, open a connection to a MongoDB server, here, running locally. We can save some space by defining the database to utilize at the same time, and requesting a handle to the default database back without needing to refer to it by name a second time. ```python client = MongoClient('mongodb://localhost/test') db = client.get_database() ``` Binding our `Account` class to a database will look up the collection name to use from the `__collection__` attribute. Alternatively you could bind directly to a specific collection. Either way, binding will automatically apply the metadata options for data access and validation and enable the `get_collection` method to provide you the correct, configured object. ```python Account.bind(db) ``` Two class methods are provided for collection management requiring awareness of our metadata: `create_collection` and `create_indexes`. Creating the collection will create any declared indexes automatically by default. For other collection-level management operations it is recommended to utilize `get_collection` and issue calls to the PyMongo API directly. ```python Account.create_collection() ``` With the class bound you can now more easily interact with your documents in the collection. # Interaction Binding the class is not strictly needed in order to interact with them. You can instantiate, manipulate, and utilize as a mapping without it. Binding does, however, allow you to easily save the result and fetch records back out. ## Record Creation When constructing an instance you may pass field values positionally as well as by name. Fields will be filled, positionally, in the order they were defined, skipping fields whose `positional` predicate is falsy. ```python alice = Account("Alice Bevan-McGregor", 'amcgregor', age=27) ``` The record has not even been persisted to the database yet and it has an identifier. This would allow you to create a batch of records, possibly with relationships, that can be committed at once. A read-only calculated property is provided to pull a creation time from the record's ObjectId creation time. ```python print(alice.id) # Already there. print(alice.created) # Creation time from ID. ``` We can now insert our record into the database. We can verify the operation (we pass the return value of the PyMongo API call back to you) by ensuring the server acknowledged the write and double-checking the record's inserted ID. This second step is for illustrative purposes and is not generally needed in the wild. ```python result = alice.insert_one() assert result.acknowledged assert result.inserted_id == alice.id ``` Using an assertion in this way, this validation will not be run in production code executed with the `-O` option passed (or `PYTHONOPTIMIZE` environment variable set) in the invocation to Python. ## Record Retrieval With a record stored in the database we can now issue queries and expect some form of result. Retrieving a record by its identifier, is simplified. As there's an oddly large amount of weird in this line, we'll break it down a bit. A call to `find_one` accepts a few different argument specifications to more flexibly serve the needs of queries simple to complex. The most basic form, taking one positional parameter, is that of querying by ID. Because our ID field is an ObjectId, it's aware that documents might have one and will pull it from a document if one is supplied. ```python alice = Account.find_one(alice) # Wait... what? print(alice.name) # Alice Bevan-McGregor ``` All of the following are equivalent to the first. You can explicitly pass in a PyMongo ObjectId, or even a string representation of one. ```python alice = Account.find_one(alice.id) ``` More complex queries can be built from comparisons directly against the fields, resulting in a `Filter` mapping. Again, ObjectId fields know how to be compared against documents which contain IDs. This comparison does not have to be inline, and you can pass in any mapping representing a MongoDB filter document if you wish. ```python alice = Account.find_one(Account.id == alice) alice = Account.find_one(Account.id == alice.id) ``` Using parametric querying, you can potentially save some typing. Multiple keyword arguments are combined using "and" logic. Note that this does not support the ability to create "or" conditions. The default comparison operator if none is specified is `eq`. You can still specify it explicitly if you wish. ```python alice = Account.find_one(id=alice) alice = Account.find_one(id=alice.id) alice = Account.find_one(id__eq=alice) alice = Account.find_one(id__eq=alice.id) ``` We can, of course, query on anything we wish and not just the ID. What happens when we try to load a record that does not exist, though? ```python eve = Account.find_one(username="eve") print(eve) # None, no record was found. ``` You can use standard Python comparison operators, bitwise operators, and several additional querying methods through class-level access to the defined fields. The result of one of these operations or method calls is a dictionary-like object that is the query, an instance of `Filter`. These may be combined through bitwise and (`&`) and bitwise or (`|`) operations. Due to Python's order of operations individual field comparisons must be wrapped in parenthesis if combining inline. It is entirely possible to save pre-constructed parts of queries for later use. It can save time (and visual clutter) to assign the document class to a short, single-character variable name to make repeated reference easier. # Fields Included with Marrow Mongo are field types covering all core types supported by MongoDB. A class model is used to define new field types, with a large amount of functionality provided by the base `Field` class itself. This base class is directly usable where the underlying field type is dynamic or not known prior to access. The `Field` class is a subclass of Marrow Schema's `Attribute` and all field instances applicable to a given `Document` class or instance are accessible using the ordered dictionary `__fields__`. ```python from marrow.mongo import Document, Field class Sample(Document): name = Field() assert 'name' in Sample.__fields__ ``` ## Name Mapping In general, basic fields accept one positional parameter: the name of the field to store data against within MongoDB. In the following example any attempt to read or write to the `field` attriute of a `MyDocument` instance will instead retrieve data from the backing document using the key `name`. If no name is specified explicitly the name of the attribute the field is assigned to is used by default. The most frequent use of this is in mapping the `_id` field from MongoDB to a less cumbersome property name. You can also pass this name using the `name` keyword argument. This may be required (if overriding the default name) for non-basic field types, and is required for complex types. ```python from marrow.mongo import Document class MyDocument(Document): field = Field('name') ``` ## Defaults & Values There are a few attributes of a field that determine what happens when an attempt is made to access a value that currently does not exist in the backing document. If no default is provided and there is no value in the backing store for the field, any attempt to read the value of the field through attribute access will result in an `AttributeError` exception. You are additionally given control over what happens when the default value is assigned to the field or the value is deleted (via `del`) from the document. ### `assign` If a default value is provided, automatically assign it to the backing document when a new instance is constructed. Default`False` ### `default` A single value to store, or a function called to generate a new value on first access (the default) or on instance construction (if `assign` is `True`). Default*No default.* ### `nullable` If `True`, will store `None`. If `False`, will store non-`None` values, or not store. (The key will be missing from the backing store.) Default`False` ### `required` This field must have a value assigned; `None` and an empty string are values. Default`False` ## Limiting Choice ### `choices` Passing either an iterable of values, or a callback producing an iterable of values, as the `choices` argument allows you to restrict the acceptable values for the field. If static, this list will be included in the validation document. In this way you can emulate an enum or a set if applied to a field encapsulated in an `Array`. The ability to restrict acceptable values this way is available to all types of field. Some, such as `Number` and its more specific subclasses, provide additional methods to restrict allowable values, such as ranges or minimums and maximums. ## Field Exclusivity ### `exclusive` Occasionally it may be useful to have two distinct fields where it is acceptable to have a value assigned to only one. We model this dependency through exclusion. By passing a `set` of field names as the `exclusive` argument you can define the fields that must not be set for the current field to be assignable. Similar to this example, if you wish to define mutual exclusivity you must define both sides of the limitation. `MyDocument` declares that if `link` is set, `mail` can not be set, and likewise the reverse. ```python class MyDocument(Document): link = Field(exclusive={'mail'}) mail = Field(exclusive={'link'}) ``` ## Data Transformation ### `transformer` As we rely on Marrow Schema we make use of its transformation and validation APIs (and objects) to allow for customization of both data ingress and egress. By default Marrow Mongo attempts to ensure the value stored behind-the-scenes matches MongoDB and BSON datatype expectations to allow for conversion-free final use. If one wanted to store Python `Decimal` objects within the database and wasn't running the latest MongoDB version which has direct support for this type, you could store them safely as strings. An easy way to accomplish this is to use Marrow Schema's `Decimal` transformer. When attempting to retrieve the value, the string stored in the database will be converted to a `Decimal` object automatically. When assigning a `Decimal` value to the attribute it will be likewise converted back to a string for storage in MongoDB. ```python from marrow.schema.transform import decimal class MyDocument(Document): field = Field(transformer=decimal) ``` ### Transformation in Field Subclasses There is a shortcut for handling transformation (when using the default transfomer) in field subclasses, used extensively by the built-in field types. When subclassing `Field` you can simply define a `to_native` and/or `to_foreign` method. These methods are passed the document the field is attached to, the name of the field, and the value being read or written. They must return either the same value, or some value after hypothetical transformation. The reason for the seeming duplication of the field information (which would otherwise be accessible via `self`) is to allow for the assignment of non-method functions, callable objects, or static methods. ```python class AwesomeField(Field): def to_native(self, doc, name, value): return value def to_foreign(self, doc, name, value): return str(value).upper() ``` ## Data Validation ### `validator` By default no data validation is performed beyond the errors that may be raised during datatype transformation for a given `Field` subclass. Any field-level configuration for validation-like features effect the generation of the MongoDB-side validation document. You can make use of custom client-side valiation within your own models by utilizing Marrow Schema validation objects. This example provides a username-based `_id` field. ```python from marrow.schema.validate.pattern import username class MyDocument(Document): id = Field('_id', validator=username) ``` ## Projection ### `project` Subclasses of `Document` provide a `__projection__` attribute containing the default set of fields to project based on field `project` predicates. Behaviour is somewhat complex; all fields excluding those marked for exclusion (`False`) are projected unless any are marked for explicit inclusion (`True`) in which case just those are. Fields whose predicates evaluate to `None` (the default) will only be included in the former case. ```python class MyDocument(Document): foo = Field() bar = Field(project=None) baz = Field(project=False) MyDocument.__projection__ == {'foo': True, 'bar': True} ``` ## Read/Write Permissions ### `read` & `write` The shortcut methods `is_readable(context=None)` and `is_writeable(context=None)` are provided to evaluate the `read` and `write` predicates, which follow a pattern similar to projection. Literal `True` and `False` are allowed as constants to represent "always" and "never". These may alternatively be callbacks (or *callable objects*) which optionally take a context argument and return `True` or `False`, or an iterable of such objects which may also return `None` to abstain from voting in the access control list (ACL). ```python class MyDocument(Document): foo = Field(write=False) MyDocument.foo.is_writeable() == False ``` ## Sorting ### `sort` Virtually identical to the `read` and `write` access permissions, the `sort` predicate follows the same rules and provides an `is_sortable(context=None)` evaluation method. # Indexes # Trait Mix-Ins # Plugin Namespaces # Decimal # Field A `Field` represents a data types storable within MongoDB and the associated machinery for access, querying, and manipulation. It is the common base class for almost all field types and can be used standalone to represent a "dynamic" field. #### Import `from marrow.mongo import Field` #### Inherits `marrow.schema:`**`Attribute`** ## Attributes #### `name` The database-side name of the field, stored internally as the metadata property `__name__`. Default calculated when assigned as a class attribute from the name given to the `Field` instance during class construction. #### `default` The default value to utilize if the field is missing from the backing store. You may assign a callback routine returning a value to utilize instead. #### `choices` The permitted set of values as a sequence; may be static or a dynamic, argumentless callback routine as per `default`. Default`None` #### `required` Must have a value assigned. `None`, an empty string, and other falsy values are acceptable. Default`False` #### `nullable` If `True`, will store `None`. If `False`, will store any non-`None` default, or remove the field from the backing store. Default`False` #### `exclusive` The set of other fields that must **not** be set for this field to be writeable. Default`None` ### Local Manipulation Define how Python-side code interacts with the stored MongoDB data values. #### `transformer` A Marrow Schema `Transformer` class to use when loading or storing values. Default`FieldTransform()` #### `validator` A Marrow Schema `Validator` class to use when validating values during assignment. Default`Validator()` #### `assign` Automatically assign the default value to the backing store when constructing a new instance or the value is found to be missing on access. Default`False` ### Predicates These are either argumentless callback routines returning, or simply the constant values: * `None`\ Interpreted as "no opinion", with the fallback being to deny or exclude. * `False` (or falsy)\ Explicitly forbid, deny, or exclude. * `True` (or truthy)\ Explicitly allow or include. These are used to restrict or define security-like behaviours. #### `positional` Permit this field to be populated through positional assignment during instantiation of its containing class. Default`True` #### `repr` Include this field in the programmers' representation, primarily utilized for REPL shells, logging, tracebacks, and other diagnostic purposes. *Protect sensitive fields from accidental exposure by assigning `False`.* Default`True` #### `project` Inlcude (or exclude) this field from the default projection. Default`None` #### `read` Permission to read values from this field. *Not internally enforced.* Default`True` #### `write` Permission to assign values to this field. *Not internally enforced.* Default`True` #### `sort` Allow sorting/ordering on this field. *Not internally enforced.* Default`True` ## Metadata Class-level metadata attributes meant for use when subclassing. #### `__name__` The database-side name of this field instance. Read-Only #### `__allowed_operators__` The permissable MongoDB filter and update operators, or hash-prefixed groups of operations. Type`set` #### `__disallowed_operators__` Specific operations may be excluded from group-based inclusion, if utilized above. Type`set` #### `__document__` A weak reference to the document the instance is bound to; automatically popualted. Read-Only #### `__foreign__` The MongoDB stored datatype, as defined by the [`$type` operator](https://docs.mongodb.com/manual/reference/operator/query/type/#available-types). Type`str` ## Usage ## Examples # Index An `Index` represents a MongoDB database index, of any kind. #### Import `from marrow.mongo import Index` #### Inherits `marrow.schema:`**`Attribute`** ## Attributes Indexes are defined via the following attributes predominantly used as [arguments to the eventual `createIndex` call](https://docs.mongodb.com/manual/reference/method/db.collection.createIndex/). The field references to include in the index are passed positionally, all other attributes may be passed as keyword arguments. Not all are utilized for each type of index, see the usage section below (or relevant MongoDB documentation for the index type) for details. #### `fields` A set of field references and their associated prefixes as strings. Initially defined as the positional arguments to the class constructor. Required #### `unique` Should the index be constructed with a unique constraint? Default`False` Official Documentation[Unique Indexes](https://docs.mongodb.com/manual/core/index-unique/) #### `background` Create the index as a background operation. Note that if this is falsy, the foreground index build will block all other operations on the database. Default`True` Official Documentation[db.collection.createIndex() Behaviours](https://docs.mongodb.com/manual/reference/method/db.collection.createIndex/#behaviors) #### `sparse` Omit from the index documents that omit the field. Default`False` Official Documentation[Sparse Indexes](https://docs.mongodb.com/manual/core/index-sparse/) #### `expire` Number of seconds after which to expire (cull/delete/remove) the record, declaring a "time-to-live" (TTL) index. Official Documentation[TTL Indexes](https://docs.mongodb.com/manual/core/index-ttl/) #### `partial` A query filter (raw, or as constructed by field comparison or use of parametric helper) to use for partial indexing. Official Documentation[Partial Indexes](https://docs.mongodb.com/manual/core/index-partial/) #### `bucket` Bucket size for use with, and only appropriate for, geoHaystack indexes. Official Documentation[geoHaystack Indexes](https://docs.mongodb.com/manual/core/geohaystack/) #### `min` The lower inclusive boundary for the longitude and latitude values. Default`-180.0` #### `max` The upper inclusive boundary for the longitude and latitude values. Default`180.0` ## Referencing Fields Fields are referenced by their "attribute path", that is, a period-separated string representing the path to that field from the top-level Document class. For example, if you define a model with a `name = Field('foo')` field, the path for this is `"name"`. If you have an embedded document, say, an `Address` with a `city` field attached as `addr = Embed(Address)`, referencing the City would be: `"addr.city"` Beyond just the reference, indexes also use prefix symbols to identify the type of index, ordering, etc. ### Index Prefixes * *No prefix* or `+` — Ascending * `-` — Descending * `@` — Geo2D * `%` — GeoHaystack * `*` — GeoSphere * `#` — Hashed * `$` — Full Text ## Usage Define your document subclass and assign instances of `Index` as attributes. The name of the attribute will be used as the MongoDB index name. To avoid potential collisions with fields or other attributes such as methods, it is recommended to prefix these attribute names with a single leading underscore. ```python from marrow.mongo import Document, Index from marrow.mongo.field import String, Integer class Person(Document): name = String() age = Integer() _age = Index('age') ``` You can then create the index by calling the `create` method of the `Index` object, passing in a collection: ```python import pymongo # Connect and retrieve a handle to the target collection. client = pymongo.MongoClient('mongodb://localhost/') db = client.test collection = db.people # Create an index in that collection. Person._age.create(collection) ``` If you are using any of the mix-in traits descendant from `Collection` (such as `Queryable`) then the `create_collection` method will, by default, also discover and create any associated indexes. ```python from marrow.mongo.trait import Queryable class WikiPage(Queryable, Document): __collection__ = 'pages' id = String('_id') content = String() _fti = Index('$content') WikiPage.bind(db) # Permit ActiveRecord-like usage. WikiPage.create_collection() # Alternatively, this can be called to create any missing indexes. # WikiPage.create_indexes() ``` ## Methods * `adapt(*args, **kw)` — create a new copy of this index with adjustments applied. Takes the same arguments as the constructor, with any new fields declared being added to the existing set. Predominantly useful for extending (and overriding) indexes declared by mix-in traits. * `create(collection, **kw)` — instruct MongoDB to construct and persist the index. If the index is not configured for `background` construction, this will block other operations on the database until complete. Additional arguments are passed through to the eventual PyMongo `collection.create_index` call. Also available via the PyMongo standard method name, `create_index`. * `drop(collection)` — instruct MongoDB to deconstruct and remove the index from the collection metadata. Also available via the PyMongo standard method name, `drop_index`. # Fields # Alias An `Alias` is a proxy to another field, potentially nested, within the same document. Utilizing an `Alias` allows class-level querying and instance-level read access, write access under most conditions, as well as optional deprecation warning generation. #### Import `from marrow.mongo.field import Alias` #### Inherits `marrow.schema:`**`Attribute`** #### Added `>=1.1.0` [Oranir](https://github.com/marrow/mongo/releases/tag/1.1.0) ## Attributes This pseudo-field **does not** inherit other field attributes. #### `path` A string reference to another field in the same containing class. See the [Usage](/reference/fields/alias#usage) section below for the structure of these references. Required #### `deprecate` Used to determine if access should issue `DeprecationWarning` messages. If truthy, a warning will be raised, and if non-boolean the string value will be included in the message. Default`False` Added`>=1.1.2` ## Usage Instantiate and assign an instance of this class during construction of a new `Document` subclass, passing the attributes in as positional (in the order seen above) or keyword arguments. This pseudo-field utilizes the Marrow Package [`traverse`](https://github.com/marrow/package#4-resolving-object-references) utility to allow it to resolve a wide variety of attributes. References to other fields may be: * The string name of a sibling attribute. Example`'id'` * The path to a descendant attribute of a sibling. Example`'address.city'` * Or involving numeric array indexes. Example`'some_array.0'` * Or involving dictionary key references. Example`'locale.en'` Paths are strings comprised of dot-separated attribute names. The search beings at the containing document, consuming path elements as we go. Each path element is first attempted as an attribute and, failing that, will attempt dictionary access. If the path element is numeric, it will be utilized as an array index. Accessing an `Alias` at the class level will resolve a Queryable for the target field, allowing filter document construction through comparison utilizing the `alias` name itself. On an instance access will retrieve or assign the value of the target field. ## Examples ### Sibling Reference As it might not be natural to refer to the user's username everywhere as `id`, especially if dereferenced from a variable, you can use an `Alias` to provide a more contextual name for the identifier. ```python class User(Document): id = String('_id') username = Alias('id') User.find_many(username__startswith="a") ``` ### Descendant Attribute of a Sibling There are situations where elevating an embedded value can be useful to, for example, shorten frequent queries or variable references in templates. ```python class Package(Document): class Address(Document): city = String() ... address = Embed(Address, assign=True) city = Alias('address.city') ``` ### Numeric Array Index Reference If you're savvy and always insert the most recent message at the beginning of the unread messages array, you can easily and semantically access the latest message using an `Alias`. ```python class Conversation(Document): class Message(Document): id = ObjectId('_id') sender = Field() message = Field() messages = Array(Embed(Message), assign=True) latest = Alias('messages.0') ``` ### Legacy Alternative Names / Deprecation Data modelling requirements change over time and there can be a lot of code referencing a given document attribute. Help identify where that access is coming from by marking old, deprecated attributes using `Alias` with an appropriate message. Added`>=1.1.2` ```python class User(Document): id = String('_id') username = Alias('id', deprecate="Username is now primary key.") ``` ## See Also * [`traverse`](https://github.com/marrow/package#4-resolving-object-references) # Array An `Array` is used to contain zero or more other values (representable using fields) in the form of a numerically indexed list. #### Import `from marrow.mongo.field import Array` #### Inherits `marrow.mongo:`**`Field`** ## Attributes This field type inherits all [`Field` attributes](https://github.com/marrow/mongo/tree/128b1ec81ec6a48e05a95c32b636deede377854b/reference/field/field.md#attributes). As a complex type, the first positional argument is always the nested field instance, other positional ordering is unaffected. #### `kind` A `Field` subclass instance, or an instance of `Field` itself if the type is dynamic. Required ## Usage Instantiate and assign an instance of this class during construction of a new `Document` subclass, passing another `Field` instance representing the type to embed as the first positional parameter. Accessing as a class attribute will return a Queryable allowing array-like filtering operations, and access as an instance attribute will return a `list` subclass containing cast values. To reduce boilerplate when costructing new document instances utilizing `Array` fields, if the `assign` attribute is truthy and no default is otherwise assigned, an empty list will be assumed and assigned, eliminating the need for armour against None or non-existant conditions. ## Examples ### Arrays of Scalar Values Tags are a very, very common storage pattern, modelled here using an `Array` of free-form `String` values. Foreign references are also common, though MongoDB itself provides no referential integrity validation. ```python class Record(Document): tags = Array(String(), assign=True) actors = Array(Reference('Account')) ``` ### Array of Embedded Documents Another principal pattern is that of an array of embedded documents, for example, an invoice with line items. ```python class Invoice(Document): class Item(Document): ... items = Array(Embed(Item), assign=True) ``` ## See Also * [`Embed`](broken://pages/-LRHtsza3puSnTgfZoua) * [`Reference`](broken://pages/-LRHtszlirbWWU4fsdBC) # Binary Binary fields store, as the name implies, raw binary data. This is the rough equivalent to a BLOB field in relational databases. The amount of storage space [**is limited**](https://docs.mongodb.com/manual/reference/limits/#bson-documents) to 16MB per document. For storage of binray data beyond this limit please utilize [GridFS](https://docs.mongodb.com/manual/core/gridfs/index.html) support. ### Import `from marrow.mongo.field import Binary` ### Inherits `marrow.mongo:`**`Field`** ## Attributes This field type inherits all [`Field` attributes](https://github.com/marrow/mongo/tree/128b1ec81ec6a48e05a95c32b636deede377854b/reference/field/field.md#attributes) and represents a singular, scalar binary string value. It has no specific configuration options. ## Usage Instantiate and assign an instance of this class during construction of a new `Document` subclass. Accessing as a class attribute will return a Queryable allowing binary string filtering operations, and access as an instance attribute will return a `bytes` cast value. ## Example Users may wish to utilize a "profile image" to identify themselves. Utilizing a Binary field (and appropriate upload limits) can facilitate this. When presenting back via HTTP, the mime type would be useful to track; see GridFS for this capability as well. ```python class User(Document): name = String('_id') avatar = Binary() me = User("amcgregor") with open('avatar.png', 'rb') as fh: me.avatar = fh.read() me.insert_one() ``` ## See Also * [`String`](/reference/fields/string) # Boolean Boolean fields store boolean values, as expected. It also provides convienence for accepting *truthy* or *falsy* values. See the usage section for more details. ### Import `from marrow.mongo.field import Boolean` ### Inherits `marrow.mongo:`**`Field`** ## Attributes This field type inherits all [`Field` attributes](https://github.com/marrow/mongo/tree/128b1ec81ec6a48e05a95c32b636deede377854b/reference/field/field.md#attributes) and represents a singular, scalar boolean value. In addition to the storage and retrieval of pure `True` and `False` values, the field configuration defines allowable "truthy" values (to cast to `True`) and "falsy" values (to cast to `False`). ### `truthy` Values to interpret as `True`, storing `True` if they are assigned. Default`('true', 't', 'yes', 'y', 'on', '1', True)` Added`>=1.1.3` ### `falsy` Values to interpret as `False`, storing `False` if they are assigned. Default`('false', 'f', 'no', 'n', 'off', '0', False)` Added`>=1.1.3` In versions prior to 1.1.3 the "truthy" and "falsy" values are hardcoded at the defaults presented above. ## Usage Instantiate and assign an instance of this class during construction of a new `Document` subclass. Accessing as a class attribute will return a Queryable allowing filtering operations, and access as an instance attribute will return a `bool` cast value. Assignment of any value matched by the `truthy` iterable (via `in` comparison) will store `True`, and likewise with the `falsy` iterable storing `False`. Additionally, if an attempt is made to assign a non-boolean, non-string value, the value will be passed through `bool()` conversion prior to storage, allowing use of objects which define their own `__nonzero__`/`__bool__` methods. # Date Date fields store `datetime` values. Times are always stored in UTC, though with appropriate support packages installed (`pytz` and/or `tzlocal`) this can include timezone support. #### Import `from marrow.mongo.field import Date` #### Inherits `marrow.mongo:`**`Field`** #### Available `>=1.1.2` ## Attributes This field type inherits all [`Field` attributes](https://github.com/marrow/mongo/tree/128b1ec81ec6a48e05a95c32b636deede377854b/reference/field/field.md#attributes) and represents a singular, scalar date/time value. #### `naive` Timezone to interpret naive \`datetime\` objects as utilizing. Default`utc` Added`>=1.1.3` #### `tz` Timezone to cast to when retrieving from the database. Default`()` Added`>=1.1.3` Timezone references as utilized by the above may be any of: * The constant string `"naive"`, resulting in no timezone transformation or alteration of the `tzinfo` attribute at all. * The constant string `"local"`, auto-detecting the host's timezone, requiring the package `localtz` be installed. This is most useful if you use `datetime.now()` instead of `datetime.utcnow()`—please consider updating your code to utilize the UTC variant in preference to this. * A `tzinfo` object, such as those provided by the `pytz` package. Any use of timezone awareness will require the `pytz` package be installed, as Python's built-in `tzinfo` objects suffer a number of issues. Note also that use of timezones comes with a performance penalty. ## Usage Instantiate and assign an instance of this class during construction of a new `Document` subclass. Accessing as a class attribute will return a Queryable allowing filtering operations, and access as an instance attribute will return a `datetime` cast value. Date fields are highly aware of date-like objects and how to apply them. For example, you may provide any of the following in place of a pure `datetime` value: * Any `MutableMapping` instance (such as a `dict` or `Document` instance) with an `_id` key whose value is an `ObjectId`. The date/time value will be pulled automatically from the `_id.generation_time`. * A bare BSON `ObjectId` instance, behaving as above. * A `datetime.timedelta` instance whose value will be immediately applied (added to) the result of `datetime.utcnow()`. * A `datetime` instance. ## Examples ### Typecasting Behaviour and Querying A key reason for the above typecasting allowances are to permit natural comparison against those types of objects as admittedly, it'll be unlikely you'll need to populate a date from the ID of a record. Given a Date field named `modified`, you can identify all documents modified in the last 30 days easily and without performing date math yourself: (remembering that the value being queried for becomes static after that comparison) ```python query = Asset.modified >= timedelta(days=-30) Asset.find(query) ``` # Double # Embed # Integer # Link A Link field type is provided to offer a way to store and retrieve well-formed URI (URL) while optionally restricting the allowable schemes, or protocols. Internally these values are stored as strings after normalization through the `URI` datatype, and on access provide that URI instance. ### Import `from marrow.mongo.field import Link` ### Inherits `marrow.mongo.field:`**`String`** ## Attributes This field type inherits all [`Field` attributes](https://github.com/marrow/mongo/tree/128b1ec81ec6a48e05a95c32b636deede377854b/reference/field/field.md#attributes) and represents a singular, scalar text value. ## Usage Instantiate and assign an instance of this class during construction of a new `Document` subclass. Accessing as a class attribute will return a Queryable allowing filtering operations, and access as an instance attribute will return a `URI` cast value. Assignment of any value matched by the `truthy` iterable (via `in` comparison) will store `True`, and likewise with the `falsy` iterable storing `False`. Additionally, if an attempt is made to assign a non-boolean, non-string value, the value will be passed through `bool()` conversion prior to storage, allowing use of objects which define their own `__nonzero__`/`__bool__` methods. # Long # Mapping # Markdown # Number # ObjectId # Path # Period # Plugin # Reference # Regex # Set # String String fields store Unicode text, utilizing the native Unicode representation for your version of Python. (On Python 2: `unicode`, on Python 3: `str`.) ### Import `from marrow.mongo.field import String` ### Inherits `marrow.mongo:`**`Field`** ## Attributes This field type inherits all [`Field` attributes](https://github.com/marrow/mongo/tree/128b1ec81ec6a48e05a95c32b636deede377854b/reference/field/field.md#attributes) and represents a singular, scalar Unicode string value. In addition to the basic attributes, String fields can automatically strip extraneous whitespace on assignment or perform case normalization, e.g. automatic execution of `str.upper()`, `str.lower()`, or `str.title()`. ### `strip` Either the boolean literal `True` or a string representing the argument to `str.strip`, that is, the characters to strip. Default`False` Added`>=1.1.1` ### `case` Truthy values (or the string literal `"u"` or `"upper"`) will request uppercase normalization, falsy values (or the literals `"l"` or `"lower"`) request lowercase normalization, or the literal `"title"` can be used to request title case. Default`None` Added`>=1.1.1` ## Usage Instantiate and assign an instance of this class during construction of a new `Document` subclass. Accessing as a class attribute will return a Queryable allowing string filtering operations, and access as an instance attribute will return a `str` (or `unicode` on Python 2) cast value. ## See Also * [`Binary`](/reference/fields/binary) # TTL # Timestamp # Parametric # F (Filter) # P (Project) # S (Sort) # U (Update) # Query # Ops # Q # Traits # Collection ## Metadata ### Collection Binding #### `__bound__` The PyMongo Collection instance this class is bound to, or `None` if not bound. Primarily meant to be used as a truthy value; utilize `.get_collection()` to acquire a handle to the PyMongo Collection if intended for use. Default`None` #### `__collection__` The string name of the collection to bind to. Can be used as a truthy value to identify if a `Document` class is top-level or not. Default`None` #### `__projection__` A **read-only** calculated property generated at class construction time identifying the default projection to utilize. This is derived from the available fields and their `project` predicates. ### Data Access Options #### `__read_preference__` The default read preference to utilize when binding. Must be an appropriate attribute value of the PyMongo [`ReadPreference`](http://api.mongodb.com/python/current/api/pymongo/read_preferences.html#pymongo.read_preferences.ReadPreference) object or customized instance of a [`read_preferences`](http://api.mongodb.com/python/current/api/pymongo/read_preferences.html#pymongo.read_preferences.ReadPreference) class. Default`ReadPreference.PRIMARY` #### `__read_concern__` The read concern (level of isolation) to utilize when binding. Must be a PyMongo [`ReadConcern`](http://api.mongodb.com/python/current/api/pymongo/read_concern.html) instance. Default`ReadConcern(level=None)` #### `__write_concern__` The default write concern (level of confirmation) to utilize when binding. Must be a PyMongo [`WriteConcern`](http://api.mongodb.com/python/current/api/pymongo/write_concern.html) instance. Default`WriteConcern(w=1, wtimeout=None, j=None, fsync=None)` ### Storage Options #### `__collection__` Default`None` #### `__collection__` Default`None` #### `__collection__` Default`None` #### `__collection__` Default`None` # Derived # Expires # Heirarchical # Identified # Localized # Lockable # Published # Stateful # Queryable # Utilities # Capped Collections # Geospatial # Logging # Colophon The Marrow Mongo Document Mapper is built on the shoulders of giants: * [Python](https://www.python.org/) programming language and standard library. * [MongoDB](https://www.mongodb.com/) document database and [PyMongo](https://api.mongodb.com/python/current/). * Documentation processing and hosting provided by [GitBook](http://www.gitbook.com/). * Much inspiration for the Contributing section provided by [Contributing Guides: A Template](https://github.com/nayafia/contributing-template/). * Communication and collaboration provided by [GitHub](https://www.github.com/) and the [Freenode IRC Network](http://freenode.net). * Service hosting graciously provided by [Clever Cloud](https://www.clever-cloud.com/). # License The [**Marrow Mongo Document Mapper**](https://mongo.webcore.io/) is Copyright © 2016 [Alice Bevan-McGregor](https://github.com/amcgregor/) and [contributors](https://github.com/marrow/mongo/graphs/contributors). Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NON-INFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. # History ## 1.1.2 [*Enhancement Release*](https://github.com/marrow/mongo/releases/tag/1.1.2) `2017-09-13` **Noaphiel** General corrections and changes: * Array field could not be filtered as not-equal. [#43](https://github.com/marrow/mongo/issues/43) * The new `Lockable` trait implements mutex lock behaviour at the document level. [#47](https://github.com/marrow/mongo/issues/47) * Package build and automated testing adjustments including expanded build matrixes, Bandit exclusions, and multiple MongoDB versions. * Dead code removal. * Adaption to future reserved word use; `await` will be reserved in Python 3.7+. * Corrections for certain edge cases involving casting of `None` values. Backwards incompatible changes: * Due to the reservedness of `await` mentioned above, its use will raise an error. Use `wait` instead. Field types now available: * `Link` for the storage of URI values such as HTTP URLs, `mailto:`, `tel:`, etc. [#45](https://github.com/marrow/mongo/issues/45) [#39](https://github.com/marrow/mongo/issues/39) * `Mapping` field to automatically perform read-only translation of a keyed list of embedded documents into a dictionary. [#46](https://github.com/marrow/mongo/issues/46) * `Set` field will utilize a true `set` instance Python-side. Field enhancements: * Fields may now be excluded from positional instantiation. [#28](https://github.com/marrow/mongo/issues/28) * Fields may now be adapted / mutated to specialize when inheriting without complete replacement. [#38](https://github.com/marrow/mongo/issues/38) * `Alias` fields may now trigger deprecation warnings if requested. [#48](https://github.com/marrow/mongo/issues/48) * `Date` fields are now timezone aware if `pytz` is installed, and able to intelligently utilize the server-local timezone if `tzlocal` is installed. (Or just utilize Marrow Mongo's `tz` installation flag.) [#51](https://github.com/marrow/mongo/issues/51) * `PluginReference` can now perform simple search and replace in Python import references, allowing for mapping of old import paths to new ones during code refactoring. [#49](https://github.com/marrow/mongo/issues/49) ## 1.1.1 [*Refinement Release*](https://github.com/marrow/mongo/releases/tag/1.1.1) `2017-05-17` **Mirthra** **Please note that due to Pypi stupidity, version** `1.1.1.1` **there is actually** `1.1.1`**.** New or updated in this release: * Removal of diagnostic information and updated testing/commit configurations, improving commit performance and bumping Pypy3 versions. * Correction of ABC participation (and missing shallow copy method) for Pypy use of query fragments. [#32](https://github.com/marrow/mongo/issues/32) * Corrected `$regex` generation. * Collation support. * Passing an existing document (with `_id` key) to an `ObjectId` field will utilize the ID provided therein. [#20](https://github.com/marrow/mongo/issues/20) * Enhanced `String` field capabilities to include stripping and case conversion. [#33](https://github.com/marrow/mongo/issues/33) * Shared `utcnow` helper function. * Improved documentation coverage. * Improved generalized programers' representations. * Improved query fragment merging. * Corrected Reference field behaviours. * Dead code removal. * Updated `Array` and `Embed` field default value handling to reduce boilerplate. **Potentially backwards-incompatible changes:** * Simplification to only support a single referenced kind in complex fields such as `Array` and `Embed`. As multi-kind support was not fully implemented, this should not disrupt much. New fields, including: * `Decimal` — [#23](https://github.com/marrow/mongo/issues/23) * `Period` — Storage of dates rounded (floor) to their nearest period. * `Markdown` — Rich storage of Markdown textual content. [#34](https://github.com/marrow/mongo/issues/34) * `Path` — Store a PurePosixPath as a string. [#35](https://github.com/marrow/mongo/issues/35) Traits are new, see #26, including: * `Collection` — Isolating collection management semantics from the core `Document` class. * `Derived` — Isolating subclass management and loading from the core `Document` class. * `Expires` — Automated inclusion of `TTL` (time-to-live) field and index definitions, including expiry check on load. * `Identified` — Isolation of primary key management from core `Document` class. * `Localized` — Management of contained localizable top-level document content. * `Published` — Management of publication/retraction and dedicated creation/modification times. * `Queryable` — Encapsulation of collection-level record management. (**Not** an Active Record pattern.) ## 1.1.0 [*Feature Release*](https://github.com/marrow/mongo/releases/tag/1.1.0) `2016-11-27` **Oranir** * Add Landscape.io integration. * Improve overall code health. [#14](https://github.com/marrow/mongo/issues/14) * Added missing project metadata. * Updated installation documentation. 81e7702 * Remove dependency on `pytz`. 815a74a * Removed our own `compat` module; schema already has a sufficient one. * Allow for `Reference` fields to cache data they reference. [#8](https://github.com/marrow/mongo/issues/8) * `Array` & `Embed` dereferencing + `Alias` pseudo-field support. [#12](https://github.com/marrow/mongo/issues/12) * Ability to dereference `Array` and `Embed` subfield values when querying through class attribute access. * Added `Alias` pseudo-field to allow the creation of shortcuts for value retrieval and assignment (via instance attribute access) and querying (through class attribute access). * `Array` and `Embed` now persist their typecasting within `__data__`, to preserve changes to nested values. (This is generally safe, however do not utilize `PluginReference` as an embeddable kind.) * Allow for fields to be combined, not just query documents. [#11](https://github.com/marrow/mongo/issues/11) * Field references (`Q` instances generated through class-based attribute access of fields) may now be combined to save time in queries involving multiple fields being compared against the same value. * Parameterized filter, sort, projection and updates. [#4](https://github.com/marrow/mongo/issues/4) * Addition of `~` inversion / `$not` support on `Ops`. * Split `Ops` types. * Ensure Document uses `odict`. * GeoJSON and geographic querying support. [#6](https://github.com/marrow/mongo/issues/6) * Added Document types: * `GeoJSON` * `GeoJSONCoord` * `Point` * `LineString` * `Polygon` * `MultiPoint` * `MultiLineString` * `MultiPolygon` * `GeometryCollection` * Added field query operators: * `near` * `intersects` * `within` * Added parametric filter operators: * `near` * `within` * `within_box` * `within_polygon` * `within_center` * `within_sphere` * `intersects` * Ability to perform certain collection-level operations. [#17](https://github.com/marrow/mongo/issues/17) * Added Document class methods: * `create_collection` * `get_collection` * `create_indexes` * Added the following Document class attributes to control collection settings: * `__collection__` - the name of the collection to use * `__read_preference__` - default ReadPreference * `__read_concern__` - default ReadConcern * `__write_concern__` - default WriteConcern * `__capped__` - the size, in bytes, to allocate as a capped collection * `__capped_count__` - additionally limit the number of records * `__engine__` - override storage engine options * `__validate__` - one of 'off' (the default), 'strict', or 'moderate'. ## 1.0.0 [*Initial Release*](https://github.com/marrow/mongo/releases/tag/1.0.0) `2016-11-21` **Turmiel** * Initial release of basic field mapping functionality.