OData REST API in JAVA with projection

It is possible to make something like that in JAVA? If yes, how? (With a sample)

Consider using the http://olingo.apache.org/ or odata4j

I make a sample in .NET that expose a REST API in OData format (to give my consumers the ability to filter, select, sort, etc using a standardized protocol).

Consideration: I expose Models different from the ones from a ORM (Avoiding exposing the DB and giving the possibility to control/restrict the consumer's query). Two User class: Models.User and DAL.User

Sample:

public IHttpActionResult GetUsers(ODataQueryOptions<Models.User> queryOptions)
{
    //Get the IQueryable from DB/Repository
    DAL.UsersDAO usersDAO = new DAL.UsersDAO();
    IQueryable<DAL.User> usersQueryable = usersDAO.GetUsers();

    //Make the projection from the 'User of Project DAL' to the 'User of ODataProject'
    IQueryable<Models.User> usersProjected = usersQueryable.Select(user => new Models.User()
    {
        Id = user.Id,
        Name = user.Name,
        Gender = user.Gender
    });

    //At this point, the query was not executed yet. And with that, it is possible to add new Filters
    //like the ones send by the client or even some rules from Business Logic
    //(ex: user only see other users from his country, etc)

    //Appling the queryOptions requested by the consumer
    IQueryable usersWithOptionsApplied = queryOptions.ApplyTo(usersProjected);
    IQueryable<Models.User> usersToExecute = usersWithOptionsApplied as IQueryable<Models.User>;

    //Execute the Query against the Database/Repository/Whatever with all the filters
    IEnumerable<Models.User> usersExecuted = usersToExecute.ToList();

    return Ok(usersExecuted);
}

The key point's are:

1 - The possibility to build a query (get the builder from the Database/Repository/Whatever)

2 - Project the query to the exposed Model (not the one from ORM)

3 - Apply the filters send from the user to the OData REST API (queryOptions)

The sample (in .NET) i uploaded here: http://multiupload.biz/2meagoxw2boa

I really apreciate any atempting to do that. A proof of concept in attempting to use OData as a standard way to cross-platform technologies.

Solution

I was really pleased to read your question since it's a subject I work on for several months now and I hope that I could provide as an open-source project. I'll try to make a concise answer but there are a lot of things to tell ;-)

I use Olingo for my proof of concept and ElasticSearch for my back-end but I have in mind an open solution for any backend (both SQL and noSQL).

There are two main parts:

Metadata configuration. Olingo provides an entity EdmProvider that is responsible for providing the metadata of the managed entities to the library. This one is called during request handling to route the request to the right element processing.

There are two cases at this level. Either you manually configure this element, either you try to auto configure by autodetecting backend structures. For the first one, we need to extend the class EdmProvider which is abstract. I introduce intermediate metadata that the custom EdmProvider will be based on since some hints are required to determine the right way to implement request (for example with ElasticSearch, parent / child relations, ...). Here is a sample below of configuring manually intermediate metadata:
```
MetadataBuilder builder = new MetadataBuilder();
builder.setNamespaces(Arrays.asList(new String[] { "odata" }));
builder.setValidator(validator);

TargetEntityType personDetailsAddressType
        = builder.addTargetComplexType("odata", 
                             "personDetailsAddress");
personDetailsAddressType.addField("street", "Edm.String");
personDetailsAddressType.addField("city", "Edm.String");
personDetailsAddressType.addField("state", "Edm.String");
personDetailsAddressType.addField("zipCode", "Edm.String");
personDetailsAddressType.addField("country", "Edm.String");

TargetEntityType personDetailsType
        = builder.addTargetEntityType(
                           "odata", "personDetails");
personDetailsType.addPkField("personId", "Edm.Int32");
personDetailsType.addField("age", "Edm.Int32");
personDetailsType.addField("gender", "Edm.Boolean");
personDetailsType.addField("phone", "Edm.String");
personDetailsType.addField(
             "address", "odata.personDetailsAddress");
```
The second approach isn't always possible since the backend doesn't necessary provide all the necessary metadata. In the case of ElasticSearch, we need to add metadata within the type mapping to support it.

Now we have this, we can focus on the request processing.
Request handling Olingo allows to handle requests based on processors. In fact, the library will route requests to processor of a type that can handle the request. For example, if you want to do something on an entity, a processor that implements EntityCollectionProcessor, CountEntityCollectionProcessor and / or EntityProcessor will be selected and used. The right method of the interfaces will be called then. It's the same for properties, ...

So we need to implement processors that will adapt requests and interact with the target backend. At this level, there is a lot of plumbing (using Olingo serializer / deserializer, build context URL, eventually extract parameters, ...) and a good approach seems to implement a generic layer to base on. The latter is responsible to execute operations on the backend (read, write, query, and so on) and also handle the conversion between the types of Olingo (Entity, Property, ) and the elements used by the backend driver (in the case of ElasticSearch, raw objects, hits - see http://www.elastic.co/guide/en/elasticsearch/client/java-api/current/client.html).

So if we need an indirection level between the exposed OData model and the backend schema, we need to implement a mapping between them at both metadata and request levels described above. This allows not to have exactly the same names for entities and their properties.

Regarding filters, we can easily access them within Olingo using the class UriInfo (see methods getFilterOption, getSelectOption, getExpandOption, getOrderByOption, getSkipOption and getTopOption), as described below within a processor:

@Override
public void readEntityCollection(final ODataRequest request,
              ODataResponse response, final UriInfo uriInfo,
              final ContentType requestedContentType)
              throws ODataApplicationException, SerializerException {
    (...)
    EntitySet entitySet = dataProvider.readEntitySet(edmEntitySet,
                uriInfo.getFilterOption(), uriInfo.getSelectOption(),
                uriInfo.getExpandOption(), uriInfo.getOrderByOption(),
                uriInfo.getSkipOption(), uriInfo.getTopOption());

    (...)
}

All the hints can be passed then to the element responsible to create requests on the backend. Here is a sample in the case of ElasticSearch (notice that the class QueryBuilder is the factory of ElasticSearch Java to build queries):

QueryBuilder queryBuilder = QueryBuilders.matchAllQuery();
if (topOption!=null && skipOption!=null) {
    requestBuilder.setFrom(skipOption.getValue())
                  .setSize(topOption.getValue());
}

Queries with the query parameter $filter is a bit tedious since we need to translate the initial query into a one to the target backend. On the class FilterOption, we have access to an instance of Expression that allows to visit the expression. The following code describes a simplified way to build an ElasticSearch query based on this class:

Expression expression = filterOption.getExpression();
QueryBuilder queryBuilder
           = expression.accept(new ExpressionVisitor() {
    (...)

    @Override
    public Object visitBinaryOperator(
              BinaryOperatorKind operator, Object left,
              Object right) throws ExpressionVisitException,
                                  ODataApplicationException {
        String fieldName = (String)left;
        // Simplification but not really clean code ;-)
        String value = ((String)right).substring(
                          1, right.length() - 1);
        return QueryBuilders.termQuery((String) left, right);
    }

    @Override
    public Object visitLiteral(String literal)
                throws ExpressionVisitException,
                ODataApplicationException {
        return literal;
    }

    @Override
    public Object visitMember(UriInfoResource member)
                throws ExpressionVisitException,
                ODataApplicationException {
        UriResourcePrimitiveProperty property
                   = (UriResourcePrimitiveProperty)
                          member.getUriResourceParts().get(0);
        return property.getProperty().getName();
    }
}

If we use the value description eq 'test' within the query parameter $filter, we will have something like that:

>> visitMember - member = [description]
>> visitLiteral - literal = 'test'
>> visitBinaryOperator - operator = eq, left = description, right = 'test'

Another tricky part consists in the way to handle navigation properties and eventually denormalization of the data in the backend. I think that it's a bit out of the scope of your question.

Feel free to ask me if something isn't clear or / and if you want more details.

Hope it helps you, Thierry