My dating app has this function where I rank user based on an algorithm which takes their last login date, and some other ‘factors’.

I thought this was gonna be easy. Well, it would’ve been if I wanted to take the entire collection to the client, the use JS to calculate the field, sort, and display to user.

Problem is, I can’t load the entire collection. That’s thousands of users. I have to sort BEFORE I return the result. Which means I have to sort at the database level before I return the results, which was quite challenging.

How-to:

First the context is, I have a ‘paginated’ with a load-more button page, which uses template-level subscription to reactively subscribe and retrieve data incrementally (for fast loading!!):

Template.SearchMain.onCreated(function() {
  var self = this;

  // initialize or reset the limit variables
  Session.set('limit', 9);

  self.autorun(function() {
    // make sure user is initialized already
    var user = Meteor.user;
    if (user) {
      // set query based on profile.ideal, and sorting based on last login and online status
      // return pipeline object ready to be ran through aggregate
      var pipeline = SearchesHelper.searchUsersPipeline();

      // make sure it did return before proceeding
      if (pipeline) {
        pipeline.push({ $limit: Session.get('limit') });
        // make subscription based on options
        self.subscriptionHandle = self.subscribe('searchUsers', pipeline);
      }

      Tracker.afterFlush(function() {
        $('#loadingUsers').waitMe({ text: '載入中...', bg: 'transparent'});
      });
    }
  });
});

The pipeline is where it got challenging:

SearchesHelper.searchUsersPipeline = function() {
  var user = Meteor.user();

  if (user) {
    var yearNow = new Date().getFullYear();
    var earliestDob = new Date(new Date().setFullYear(yearNow - user.profile.ideal.ageTo - 1));
    var latestDob = new Date(new Date().setFullYear(yearNow - user.profile.ideal.ageFrom));

    var query = {
      // opposite gender of current user only
      'profile.gender': user.profile.gender == 'male' ? 'female' : 'male',

      // active members only
      userStatus: 'active',

      // filter by profile.ideal.ageFrom and ageTo
      'profile.dob': {$gte: earliestDob, $lte: latestDob}
    };

    // filter out users already passed on, or got passed on, or matches that were cancelled
    var userId = user._id;
    var skipUserIds = [];
    var userIsSenderOrReceiverQuery = { $or: [{receiverId: userId}, {senderId: userId}] };
    var matchIsPassedOrCancelledQuery = { $or: [{passed: true}, {cancelled: true}] };
    var skipMatchesCursor = Matches.find({
      $and: [userIsSenderOrReceiverQuery, matchIsPassedOrCancelledQuery]
    });
    // pass the sender if user is receiver, vice versa
    skipMatchesCursor.forEach(function(match) {
      userId == match.receiverId ? skipUserIds.push(match.senderId) : skipUserIds.push(match.receiverId);
    });
    // exclude user self as well
    skipUserIds.push(userId);
    // skip all users in skipUserIds array
    query['_id'] = {$nin: skipUserIds};

    // filter city
    if (user.profile.ideal.city != 'city_nopref')
      query['profile.city'] = user.profile.ideal.city;

    // filter education level
    if (user.profile.ideal.education != 'education_nopref') {
      var educationLevels = ['primary', 'secondary', 'associate', 'bachelor', 'masters', 'doctors'];
      var idealLevel = educationLevels.indexOf(user.profile.ideal.education);
      var acceptEduArray = educationLevels.slice(idealLevel, educationLevels.length);
      query['profile.education'] = { $in: acceptEduArray };
    }

    // filter children
    if (user.profile.ideal.children != 'children_nopref')
      query['profile.children'] = 'none';

    // filter smoke
    var smokingLevels = ['never_smoke', 'quit_smoke', 'seldom_smoke', 'sometime_smoke', 'always_smoke'];
    if (user.profile.ideal.smoke == 'sometime_smoke') {
      query['profile.smoke'] = { $ne: smokingLevels[4] };
    }
    else if (user.profile.ideal.smoke == 'no_smoke') {
      query['profile.smoke'] = { $in: [smokingLevels[0], smokingLevels[1]] }
    }

    // filter drink
    var drinkingLevels = ['never_drink', 'quit_drink', 'seldom_drink', 'sometime_drink', 'always_drink'];
    if (user.profile.ideal.drink == 'sometime_drink') {
      query['profile.drink'] = { $ne: drinkingLevels[4] };
    }
    else if (user.profile.ideal.drink == 'no_drink') {
      query['profile.drink'] = { $in: [drinkingLevels[0], drinkingLevels[1]] }
    }

    // set how many days would substract one point
    var msPerScore = 86400000 * 2;
    // Diff between now and lastLogin, in MS
    var dateDiffOperator = { $subtract: [ new Date(), "$status.lastLogin.date" ] };
    // return dateScore from lastLogin date
    var dateScoreOperator = { $subtract: [10, { $divide: [dateDiffOperator, msPerScore] }] };
    // return totalScore from weighted geometric average of adminScore and DateScore
    var totalScoreOperator = { $multiply: [dateScoreOperator, "$adminScore"]};

    var project = {
      username: 1,
      profile: 1,
      status: 1,
      userStatus: 1,
      todayMatch: 1,
      starEnd: 1,
      adminScore: 1,
      dateScore: dateScoreOperator,
      totalScore: totalScoreOperator
    };

    var sort = { 'totalScore': -1 };

    return [
      { $match: query },
      { $project: project },
      { $sort: sort }
    ]
  }
};

Basically the logic is this.

First the query, I filter out all the search results based on user’s preference (if they want a partner that is certain age range, doesn’t smoke, etc).

Then I have to make a projection. This is the most challenging part. I have to dig through mongoDB’s API to find out their arithmetic APIs. Also apparently some didn’t work on my version of meteor mongo, so things like floor, square, which I wanted to use, was not available. Perhaps that’s available by now. Anyways I decided to get by with basic multiplications and subtractions.

Once I calculated the ‘dateScore’ and ‘totalScore’ field, then the easiest part and the end, sort by totalScore in descending order.

Packages:

meteorhacks:aggregate (it’s needed to use the mongo aggregate function within Meteor. Not sure if that’s still needed in the future, maybe meteor will integrate that into the meteor mongo)

Challenges:

This task was really stretching my mongoDB abilities. I had taken the MongoDB University 12-week course, which went through all of this, but putting it in practice and learning is completely different.

Nevertheless I was glad that I knew about the mongoDB aggregation pipeline, and know how to dig deeper to find the stuff I needed to complete this task.