T-tests [52] at P-value = 0.1, rejecting the null ^ ^ hypothesis H0 that x are consistent across the databases, H0 : x ?0. Thus, x ij express the consistency of the database i with the other databases, along the j-th network measure. Note also that the absolute values of individual residuals j^j imply a ranking R over the databases, where x the database with the lowest j^j has rank one, the second one has rank two and the one with x the largest j^j has rank N. x T0901317MedChemExpress T0901317 Identifying independent network measures. Denote rij to be the Pearson product-moment ^ correlation coefficient of the residuals x for i-th and j-th network measure over all databases. Spearman rank correlation coefficient ij is defined as the Pearson coefficient of the ranks R for i-th and j-th statistics. Under the null hypothesis of statistical independence of i-th and j-th statistics, H0:ij = 0, adjusted Fisher transformation [53]: pffiffiffiffiffiffiffiffiffiffiffiffi N ?3 1 ?rij ln 2 1 ?rij ??PLOS ONE | DOI:10.1371/journal.pone.0127390 May 18,12 /Consistency of Databasesapproximately follows a standard normal distribution. Pairwise independence of the selected network measures is thus confirmed by the independent two-tailed z-tests. This gives 13 independent measures for directed, and 7 independent measures for undirected networks, as shown in the Fig 3. Furthermore, Friedman rank test [54] confirms that chosen set of measures exhibits significant internal differences, as to still be informative on the databases (see below). Ranking of databases. Significant inconsistencies between the databases are exposed using the methodology introduced for comparing classification algorithms over multiple data sets [55]. Denote Ri to be the mean rank of i-th database over the fnins.2015.00094 selected measure, Ri = j Rij/K, where K is the number of independent measures K 2 7, 13. One-tailed Friedman rank test [54, 56] first verifies the null hypothesis that the databases are statistically equivalent and thus their ranks Ri should equal, H0:Ri = Rj. Under the assumption that the selected statistics are indeed independent, the Friedman testing statistic [54]: ! 2 X 12K N ?1?2 ??Ri ?4 N ?1?i has 2-distribution with N-1 degrees of freedom. By rejecting the hypothesis at P-value = 0.1, we proceed with the Nemenyi post-hoc test that reveals wcs.1183 databases whose ranks Ri differ more than the critical difference [57]: rffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi N ?1???; q 6K where q is the critical value based on the studentized range statistic [55], q = 2.59 at Pvalue = 0.1. A critical difference diagram plots the databases with no statistically significant inconsistencies in the selected statistics [55].Supporting InformationS1 File. Degree and clustering graphical profiles, continuation of network measures. Node degree and clustering profiles and distributions of all the considered networks, along with other network statistics. See Methods for interpretation and details on computation. (PDF)AcknowledgmentsAuthors thank American Physical Society and Thomson Reuters for providing the data. Thomson Reuters had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. Work supported by purchase Naramycin A Creative Core FISNM-3330-13-500033 funded by the European Union and The European Regional Development Fund, by the H2020-MSCA-ITN-2015 project COSMOS 642563, by the Slovenian Research Agency (ARRS) via programs P2-0359, P1-0383, via projects J1-5454, L7-4119,.T-tests [52] at P-value = 0.1, rejecting the null ^ ^ hypothesis H0 that x are consistent across the databases, H0 : x ?0. Thus, x ij express the consistency of the database i with the other databases, along the j-th network measure. Note also that the absolute values of individual residuals j^j imply a ranking R over the databases, where x the database with the lowest j^j has rank one, the second one has rank two and the one with x the largest j^j has rank N. x Identifying independent network measures. Denote rij to be the Pearson product-moment ^ correlation coefficient of the residuals x for i-th and j-th network measure over all databases. Spearman rank correlation coefficient ij is defined as the Pearson coefficient of the ranks R for i-th and j-th statistics. Under the null hypothesis of statistical independence of i-th and j-th statistics, H0:ij = 0, adjusted Fisher transformation [53]: pffiffiffiffiffiffiffiffiffiffiffiffi N ?3 1 ?rij ln 2 1 ?rij ??PLOS ONE | DOI:10.1371/journal.pone.0127390 May 18,12 /Consistency of Databasesapproximately follows a standard normal distribution. Pairwise independence of the selected network measures is thus confirmed by the independent two-tailed z-tests. This gives 13 independent measures for directed, and 7 independent measures for undirected networks, as shown in the Fig 3. Furthermore, Friedman rank test [54] confirms that chosen set of measures exhibits significant internal differences, as to still be informative on the databases (see below). Ranking of databases. Significant inconsistencies between the databases are exposed using the methodology introduced for comparing classification algorithms over multiple data sets [55]. Denote Ri to be the mean rank of i-th database over the fnins.2015.00094 selected measure, Ri = j Rij/K, where K is the number of independent measures K 2 7, 13. One-tailed Friedman rank test [54, 56] first verifies the null hypothesis that the databases are statistically equivalent and thus their ranks Ri should equal, H0:Ri = Rj. Under the assumption that the selected statistics are indeed independent, the Friedman testing statistic [54]: ! 2 X 12K N ?1?2 ??Ri ?4 N ?1?i has 2-distribution with N-1 degrees of freedom. By rejecting the hypothesis at P-value = 0.1, we proceed with the Nemenyi post-hoc test that reveals wcs.1183 databases whose ranks Ri differ more than the critical difference [57]: rffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi N ?1???; q 6K where q is the critical value based on the studentized range statistic [55], q = 2.59 at Pvalue = 0.1. A critical difference diagram plots the databases with no statistically significant inconsistencies in the selected statistics [55].Supporting InformationS1 File. Degree and clustering graphical profiles, continuation of network measures. Node degree and clustering profiles and distributions of all the considered networks, along with other network statistics. See Methods for interpretation and details on computation. (PDF)AcknowledgmentsAuthors thank American Physical Society and Thomson Reuters for providing the data. Thomson Reuters had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. Work supported by Creative Core FISNM-3330-13-500033 funded by the European Union and The European Regional Development Fund, by the H2020-MSCA-ITN-2015 project COSMOS 642563, by the Slovenian Research Agency (ARRS) via programs P2-0359, P1-0383, via projects J1-5454, L7-4119,.