How to configure packet manager for priority based fetch?

Owned by Rounak Nayak (Unlicensed), created
Last updated: May 31, 2021
4 min read

In our MOSIP version 1.1.5 we have introduced priority-based search in packet manager. This feature in packet manager helps us to fetch data on priority if there are multiple external sources for an applicant's data validation (these data are created in the object store in packet form). This is highly possible if a country builds a portal where residents have the feature to correct their data if their registration process in the registration processor is paused due to some missing information or document.

Scenario

Let’s assume a country is using MOSIP to register its residents and in the backend, it is validating the resident’s data and found an issue stating that there is a document missing or some data for an applicant is not correct as per the document. For such scenarios, the country can build an online system/portal (let’s say we call it, PORTAL here) using which the resident can send a correction packet to MOSIP to update his/her data before generating or allocation of his/her UIN.

Instructions

  1. First of all the country need to build a custom packet reader and writer as part of the packet manager implementation for the correction packet or use mosip’s default packet reader & writer and then configure it in application-mz.properties.

    # NOTE: Example: Here, we are planning to add a new source called PORTAL for CORRECTION packet we have mapped the req. reader and writer here. # Reader Configurations provider.packetreader.mosip=source:REGISTRATION_CLIENT,process:NEW|UPDATE|LOST,classname:io.mosip.commons.packet.impl.PacketReaderImpl provider.packetreader.resident=source:RESIDENT,process:RES_UPDATE|LOST|RES_REPRINT,classname:io.mosip.commons.packet.impl.PacketReaderImpl provider.packetreader.portal=source:PORTAL,process:CORRECTION,classname:io.mosip.commons.packet.impl.PortalPacketReaderImpl # Writer Configurations provider.packetwriter.mosip=source:REGISTRATION_CLIENT,process:NEW|UPDATE|LOST,classname:io.mosip.commons.packet.impl.PacketWriterImpl provider.packetwriter.resident=source:RESIDENT,process:RES_UPDATE|LOST|RES_REPRINT,classname:io.mosip.commons.packet.impl.PacketWriterImpl provider.packetwriter.portal=source:PORTAL,process:CORRECTION,classname:io.mosip.commons.packet.impl.PortalPacketWriterImpl

  2. The next thing that they need to configure is the priority for fetching packets. Priority in packet manager can be set using two types of configurations,

    1. Default priority: When the stage level priority is not configured, The packet manager prefers the default configurations mentioned in the application.mz property file. The packet manager makes API calls without the source & process information. The source and process is selected based on configuration defined in MOSIP’s application-mz.properties.

      # If source is not passed, packetmanager supports below default strategy, # 1.If the value for the below property is set as, 'exception' - it will throw exception. # 2.If the value for the below property is seta as, 'defaultPriority' - packet manager will use the property packetmanager.default.priority packetmanager.default.read.strategy=defaultPriority packetmanager.default.priority=source:PORTAL\/process:CORRECTION,source:REGISTRATION_CLIENT\/process:NEW|UPDATE|LOST,source:RESIDENT\/process:RES_UPDATE|RES_REPRINT

      Here source "PORTAL" and process "CORRECTION" will have top priority. Followed by source "REGISTRATION_CLIENT" and process "NEW" followed by source REGISTRATION_CLIENT and process "UPDATE" and so on.

    2. Stage-specific priority: also needs to be set for each stage. Each stage will be reading its own priority based on the ID attribute defined in stage level priority, which needs to be configured in the properties defined in registration-processor-mz.properties
      The attributes mentioned here should be the same as the values defined in the id schema.

      # To define priority for respective stages use below stage names. # packetreceiver,packetuploader,packetvalidator,qualitychecker,osivalidator,demodedupe,classification,classifier,bioauth,manualverification,uingenerator,messagesender # For example, field 'gender' has priority in uin-generator-stage. The key for uin-generator-stage is 'uingenerator'. So the key would be 'packetmanager.provider.uingenerator.gender' packetmanager.provider.uingenerator.lastName=source:PORTAL\/process:CORRECTION,source:REGISTRATION_CLIENT\/process:NEW|UPDATE|LOST,source:RESIDENT\/process:RES_UPDATE|RES_REPRINT packetmanager.provider.uingenerator.gender=source:PORTAL\/process:CORRECTION,source:REGISTRATION_CLIENT\/process:NEW|UPDATE|LOST,source:RESIDENT\/process:RES_UPDATE|RES_REPRINT packetmanager.provider.uingenerator.city=source:PORTAL\/process:CORRECTION,source:REGISTRATION_CLIENT\/process:NEW|UPDATE|LOST,source:RESIDENT\/process:RES_UPDATE|RES_REPRINT

      Key for different stages are specified below:

      1. regproc-packet-receiver-stage - packetreceiver

      2. regproc-packet-uploader-stage - packetuploader

      3. reproc-packet-validator-stage - packetvalidator

      4. regproc-quality-checker-stage - qualitychecker

      5. regproc-osi-validator-stage - osivalidator

      6. regproc-demo-dedupe-stage - demodedupe

      7. regproc-classification-stage - classification

      8. regproc-bio-dedupe-stage - classifier

      9. regproc-bio-auth-stage - bioauth

      10. regproc-manual-verification-stage - manualverification

      11. regproc-uin-generator-stage - uingenerator

      12. regproc-message-sender-stage - messagesender
        Note: For regproc-abis-handler-stage and regproc-abis-middleware-stage the key is same as regproc-biodedupe-stage

  3. After these changes are done, the packet manager and the respective stage need to be restarted.

Special Case: For Biometrics (as multiple biometrics needs to merge a single document)

There can also be a scenario where some biometrics are received from system A and some biometrics are received from system B, but at the end of the day, we might need to merge both the data points into a single CBEFF and process it. We handle this merging and processing using a special stage level configuration. Please note that biometrics merging will not work with the default configurations.

Let's assume a scenario where an individual had an accident and his right hand had been bandaged so the operator had marked his right hand as an exception and captured other biometrics. Now, the Manual Verifier (a registered officer verifying exception packets in the registration processor), finds that this is not a biometric exception case, hence requests residents to visit the center and provide biometric through a correction portal. Now in the system, we will have two packets one from the registration client and one from the correction portal. Hence the biometrics needs to be set in the respective stage as:

Points to consider

The process in the packet needs to be unique, stating the process being followed for the packet. In MOSIP by default, there are few processes defined,

  • NEW for registering an individual on-field using a registration client

  • UPDATE for updating details of a UIN holder on the field using a registration client

  • LOST for finding a resident’s UIN by collecting their biometrics using a registration client

  • RES_UPDATE for updating details of a UIN holder when a request comes from Resident portal

The source tells us about the source of origin, from where the packet is created. Registration-Client is considered as the source. NEW, UPDATE or LOST are different processes, whereas Resident Portal is considered for process as UPDATE and source as Resident portal.