Friday, February 19, 2010

Migrating a Spring/Hibernate application to MongoDB - Part 1

Backgorund

For the past few years ORM have been the de facto solution for bridging the gap between object oriented programming languages and relational databases.Well, most of the developers using ORM care about writing less persistence code and SQL than they care about the object-relational impedance mismatch. As time passed and more experience gained, some people started to claim that maybe ORM is not the best solution available.
Another option for storing your objects, which have been there for quite some time, are non-SQL databases. With recent explosion of non-relational databases and the NoSQL movement ("Not Only SQL") this option is becoming more and more viable.

There are a lot of examples showing how to develop a new application based on a non-relational database. But what if you already have an application using a relational database + ORM  and you want to migrate it to a non-relational database?

In the next few posts I will try to suggest a migration path for a Spring/Hibernate (JPA) application to MongoDB.
MongoDB is a scalable, high-performance, open source, schema-free, document-oriented database and is one of the interesting non-relational databases available today (together with Cassandra, HBase, Redis and others).

The application

The example application is a very simple blogging engine implemented using the Spring/Hibernate (JPA) stack.
The two main entities are Blogger and BlogPost. There are data access objects (DAO) with matching interfaces for both entities.

Setting up and connecting to MongoDB

Setting up MongoDB is a pretty simple procedure. The MongoDB quickstart and getting started pages contains all the required details.
In order to connect to MongoDB we will need to use the Mongo Java driver :

<dependency>
    <groupId>org.mongodb</groupId>
    <artifactId>mongo-java-driver</artifactId>
    <version>1.2.1</version>
<dependency>


Next step is adding a new Spring service that will provide MongoDB connections. This is a basic implementation which uses the default configuration.

3    import com.mongodb.DB;
4    import com.mongodb.Mongo;
5    import java.net.UnknownHostException;
6    
7    public class MongoService {
8        private final Mongo mongo;
9        private final DB db;
10   
11       public MongoService(final String dbName) throws UnknownHostException {
12           mongo = new Mongo(); // MongoDB server (localhost:27017) 
13           db = mongo.getDB(dbName); // Connect to database
14       }
15   
16       public Mongo getMongo() {
17           return mongo;
18       }
19   
20       public DB getDb() {
21           return db;
22       }
23   }


The Mongo class is responsible for the database connection and contains a connection pool. The default pool size has 10 connections per host. You can configure the pool size by using the MONGO.POOLSIZE system property or by passing a MongoOptions parameter to the Mongo constructor.
The DB class represents a logical database on the MongoDB server. We will use a database names "blog" for the blogging application.

<bean id="mongo" class="my.demo.blog.services.MongoService">
        <constructor-arg index="0" value="blog"/>
    </bean>

Entities and Documents

MongoDB stores data in collections of BSON documents. Documents may contain any number of fields of any length and type. Usually you should store documents of the same structure within collections. MongoDB collections are essentially named groupings of documents.
The Mongo Java driver provides a DBObject interface to save custom objects to the database.
The DBOject is very similar to a Map with String keys. You can put/get document element by their String key and get a list of all available keys.
In order to save our entities in MongoDB we will create an adapter which implements the DBObject interface.

public class DbObjectAdapter implements DBObject {
    private final BeanUtilsBean beanUtils;
    private final Object entity;
    private final Set<String> keySet;

    public DbObjectAdapter(Object entity) {
        if (entity == null) {
            throw new IllegalArgumentException("Entity must not be null");
        }
        if (!entity.getClass().isAnnotationPresent(Entity.class)) {
            throw new IllegalArgumentException("Entity class must have annotation javax.persistence.Entity present");
        }
        this.entity = entity;
        this.beanUtils = new BeanUtilsBean();
        this.keySet = new HashSet<String>();
        initKeySet();
    }

    @Override
    public Object put(String name, Object value) {
        try {
            beanUtils.setProperty(entity, name, value);
        } catch (Exception e) {
            return null;
        }
        return value;
    }
    
    @Override
    public Object get(String name) {
        try {
            return beanUtils.getProperty(entity, name);
        } catch (Exception e) {
            return null;
        }
    }

In order to decide which members of the entity we would like to store in MongoDB, we create and new annotation - @MongoElement - and annotate the selected getter methods.

    @MongoElement
    public String getDisplayName() {
        return displayName;
    }

The DbObjectAdapter creates the document key set by looking for the annotated methods.

    @Override
    public Set<String> keySet() {
        return keySet;
    }

    private void initKeySet() {
        final PropertyDescriptor[] descriptors = beanUtils.getPropertyUtils().getPropertyDescriptors(entity);
        for (PropertyDescriptor desc : descriptors) {
            final Method readMethod = desc.getReadMethod();
            MongoElement annotation;
            if ((annotation = readMethod.getAnnotation(MongoElement.class)) != null) {
                keySet.add(desc.getName());
            }
        }
    }

After having the DbObjectAdapter we can create a base DAO class for storing entities in MongoDB

public abstract class BaseMongoDao<S> implements BaseDao<S> {
    private MongoService mongo;
    private DBCollection collection;

    @Override
    public S find(Object id) {
        final DBObject dbObject = collection.findOne(new ObjectId((String) id));
        final DbObjectAdapter adapter = new DbObjectAdapter(getEntityClass());
        adapter.putAll(dbObject);
        return (S) adapter.getEntity();
    }

    @Override
    public void save(S entity) {
        collection.save(new DbObjectAdapter(entity));
    }

    @Autowired
    public void setMongo(MongoService mongo) {
        this.mongo = mongo;
        // use the entity class name as the collection name
        this.collection = mongo.getDb().getCollection(getEntityClass().getSimpleName());
    }

    /**
     * 
     * @return the entity class this DAO handles
     */
    public abstract Class getEntityClass();
}

Notice that there is no need to create the collections, the database creates it automatically on the first insert.

Next part

So far we've seen how to setup MongoDB and how to store our entities in it.
In the next parts of this series we will discuss the following migration topics:
  • Identifiers
  • Relations
  • Queries
  • Data migration

5 comments:

Artorius said...

pretty interesting post, looking forward for the next parts.

What are your thoughts about cassandra instead of mongodb?

Fran said...

Thank you for that interesting article. I'm waiting for the next one in the series.

Dima Gutzeit said...

The above example miss some method implementations (at least not for mongo v2 java driver). So here is a full class:


public class DbObjectAdapter implements DBObject {
private static final Logger log = LoggerFactory.getLogger(DbObjectAdapter.class);
private final BeanUtilsBean beanUtils;
private final Object entity;
private final Set keySet;
private boolean partialObject;

public DbObjectAdapter(Object entity) {
if (entity == null) {
throw new IllegalArgumentException("Entity must not be null");
}
if (!entity.getClass().isAnnotationPresent(Entity.class)) {
throw new IllegalArgumentException("Entity class must have annotation javax.persistence.Entity present");
}
this.entity = entity;
this.beanUtils = new BeanUtilsBean();
this.keySet = new HashSet();
initKeySet();
}

@Override
public Object put(String name, Object value) {
try {
beanUtils.setProperty(entity, name, value);
} catch (Exception e) {
log.error("Failed setting property", e);
}
return value;
}

@Override
public void putAll(BSONObject bsonObject) {
putAll(bsonObject.toMap());
}

@Override
public void putAll(Map map) {
try {
beanUtils.populate(entity, map);
} catch (Exception e) {
log.error("Failed populating map", e);
}
}

@Override
public Object get(String name) {
try {
return beanUtils.getProperty(entity, name);
} catch (Exception e) {
return null;
}
}

@Override
public Map toMap() {
Map map = Maps.newHashMap();
Iterator iterator = keySet.iterator();
while (iterator.hasNext()) {
String key = iterator.next();
map.put(key, get(key));
}

return map;
}

@Override
public Object removeField(String s) {
Object retVal = get(s);
keySet.remove(s);
return retVal;
}

@Override
public boolean containsKey(String s) {
return keySet.contains(s);
}

@Override
public boolean containsField(String s) {
return keySet.contains(s);
}

@Override
public Set keySet() {
return keySet;
}

private void initKeySet() {
final PropertyDescriptor[] descriptors = beanUtils.getPropertyUtils().getPropertyDescriptors(entity);
for (PropertyDescriptor desc : descriptors) {
final Method readMethod = desc.getReadMethod();
MongoElement annotation;
if ((annotation = readMethod.getAnnotation(MongoElement.class)) != null) {
keySet.add(desc.getName());
}
}
}

@Override
public void markAsPartialObject() {
partialObject = true;
}

@Override
public boolean isPartialObject() {
return partialObject;
}
}

sasajovancic said...

Nice article, i think NoSQL solutions are future.

Here is one more way to convert your project to MongoDB with JPA.
http://sasajovancic.blogspot.com/2011/06/use-jpa-with-mongodb-and-datanucleus.html

maximos said...

Thank you so much, really helpful example.
I am having trouble creating the anotation MongoElement :
"MongoElement annotation;
if ((annotation = readMethod.getAnnotation(MongoElement.class)) != null) { keySet.add(desc.getName());
}"
Eclipse complains:"MongoElement cannot be resolved to a type" and
"Bound mismatch: The generic method getAnnotation(Class) of type AnnotatedElement is not applicable for the arguments (Class). The inferred type MongoElement is not a valid substitute for the bounded parameter " any ideas? thank you in advanced