Team:SJTU-Software/Database/AssessmentModel

Assessment Model

An assessment model is optimized to judge the quality of biobricks. With a default score given to each biobrick respectively, biobricks that are related to the input keyword in the “Easy BBK” search engine can be listed in descending order of scores although users can define their own sorting order. If sorted by default scores, users can always get biobricks with high quality. Our assessment model considered 4 general properties, namely status, reliability, feedback and publication, based on 12 attributes of a biobrick. Weights of the attributes in the general properties are already optimized and fixed; weights of the 4 general properties can be adjusted by users although default weights of the properties are optimized already and recommended to users.

Evaluation criteria

Based on the description on the website of Registry of Standard Biological Parts and advice collected from our instructor, 13 attributes of biobricks are picked out as evaluation criteria in the assessment model.
Considering different aspects users frequently takes into account when choosing a biobrick, we deliberately divided these attributes into 4 properties, which are listed in Table 1.

Table 1 The four properties in the assessment model

More detailed description of the attributes in the properties is demonstrated in Table 2.

Table 2 Detailed attributes in the assessment model

Scores given to the values in each criterion

There are two types of values in all the criteria: categorical variables (such as part status, part results, etc) and continuous variables (such as used times, average stars, etc).
For categorical variables, we give reasonable scores to different values in each criterion based on the researchers' experience and description of the values.
For continuous variables, we normalize the values in those criteria using the following formula, so that all the continuous variables range from 0 to 1 like categorical variables:

where x is an original value, x^' is the normalized value. The scores given to different values in all the attributes is presented in Table 3.

Table 3 Different scores given to the different values in all the criteria

Weights given to each criterion

50 biobricks were picked out as positive samples in Table 4. The weight of each criterion are then trained and adjusted to make the biobricks in the positive samples ranking the highest in the final scores.

Table 4 Part of the positive samples in our assessment model

The weights of the criteria in each property are fixed and cannot be adjusted by users. In “Easy BBK”, users could adjust the weight of the 4 properties - Status, Reliability, Feedback, Publication - in our assessment model although optimized weights and default scores have already given to the biobricks in the assessment model. The optimized weights are presented in Table 5.

Table 5 Weights given to the attributes in the default score of the biobricks

A glance at the assessment results

Table 6 shows the first three biobricks of different types:

The distribution of default scores of all biobricks is shown as follows:

BBa_B0034 and BBa_B0031 are both RBS, ranked the 1^st and 4^th in type RBS in our assessment model respectively. The information of the two biobricks is listed as follows.

Obviously, the two biobricks are the same in many attributes, and based on the overall information, we can generally conclude that the two parts are both of good quality.

However, when it comes to the number of used times, commented times, and number of results on Google scholar, it is suggested that BBa_B0031 is less popular and has less number of studies than BBa_B0034. What’s more, on Registry, the definition of efficiency of BBa_B0034 is 1.0, which is much higher than 0.07 of BBa_B0031. This is also literally confirmed by their short description.

The conclusion above and our user studies have confirmed that the quality of biobricks is in accordance with the scores in our assessment model.