--- http_interactions: - request: method: get uri: https://api.crossref.org/works/10.1186/1471-2164-7-187 body: encoding: UTF-8 string: '' headers: Connection: - close Host: - api.crossref.org User-Agent: - http.rb/5.1.1 response: status: code: 200 message: OK headers: Date: - Sun, 12 Mar 2023 22:21:45 GMT Content-Type: - application/json Transfer-Encoding: - chunked Access-Control-Expose-Headers: - Link Access-Control-Allow-Headers: - X-Requested-With, Accept, Accept-Encoding, Accept-Charset, Accept-Language, Accept-Ranges, Cache-Control Access-Control-Allow-Origin: - "*" Server: - Jetty(9.4.40.v20210413) X-Ratelimit-Limit: - '50' X-Ratelimit-Interval: - 1s X-Api-Pool: - public X-Rate-Limit-Limit: - '50' X-Rate-Limit-Interval: - 1s Permissions-Policy: - interest-cohort=() Connection: - close body: encoding: UTF-8 string: '{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2023,2,23]],"date-time":"2023-02-23T14:27:37Z","timestamp":1677162457093},"reference-count":38,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2006,7,25]],"date-time":"2006-07-25T00:00:00Z","timestamp":1153785600000},"content-version":"tdm","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/2.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["BMC Genomics"],"published-print":{"date-parts":[[2006,12]]},"abstract":"Abstract<\/jats:title>\n \n Background<\/jats:title>\n Accurate evaluation of the quality of genomic or proteomic data and computational methods is vital to our ability to use them for formulating novel biological hypotheses and directing further experiments. There is currently no standard approach to evaluation in functional genomics. Our analysis of existing approaches shows that they are inconsistent and contain substantial functional biases that render the resulting evaluations misleading both quantitatively and qualitatively. These problems make it essentially impossible to compare computational methods or large-scale experimental datasets and also result in conclusions that generalize poorly in most biological applications.<\/jats:p>\n <\/jats:sec>\n \n Results<\/jats:title>\n We reveal issues with current evaluation methods here and suggest new approaches to evaluation that facilitate accurate and representative characterization of genomic methods and data. Specifically, we describe a functional genomics gold standard based on curation by expert biologists and demonstrate its use as an effective means of evaluation of genomic approaches. Our evaluation framework and gold standard are freely available to the community through our website.<\/jats:p>\n <\/jats:sec>\n \n Conclusion<\/jats:title>\n Proper methods for evaluating genomic data and computational approaches will determine how much we, as a community, are able to learn from the wealth of available data. We propose one possible solution to this problem here but emphasize that this topic warrants broader community discussion.<\/jats:p>\n <\/jats:sec>","DOI":"10.1186\/1471-2164-7-187","type":"journal-article","created":{"date-parts":[[2006,7,25]],"date-time":"2006-07-25T18:14:51Z","timestamp":1153851291000},"update-policy":"http:\/\/dx.doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":171,"title":["Finding function: evaluation methods for functional genomic data"],"prefix":"10.1186","volume":"7","author":[{"given":"Chad L","family":"Myers","sequence":"first","affiliation":[]},{"given":"Daniel R","family":"Barrett","sequence":"additional","affiliation":[]},{"given":"Matthew A","family":"Hibbs","sequence":"additional","affiliation":[]},{"given":"Curtis","family":"Huttenhower","sequence":"additional","affiliation":[]},{"given":"Olga G","family":"Troyanskaya","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2006,7,25]]},"reference":[{"key":"570_CR1","volume-title":"Bioinformatics","author":"Z Barutcuoglu","year":"2006","unstructured":"Barutcuoglu Z, Schapire RE, Troyanskaya OG: Hierarchical multi-label prediction of gene function. Bioinformatics. 2006"},{"issue":"Suppl 2","key":"570_CR2","doi-asserted-by":"publisher","first-page":"II42","DOI":"10.1093\/bioinformatics\/btg1058","volume":"19","author":"A Clare","year":"2003","unstructured":"Clare A, King RD: Predicting gene function in Saccharomyces cerevisiae. Bioinformatics. 2003, 19 (Suppl 2): II42-II49.","journal-title":"Bioinformatics"},{"key":"570_CR3","first-page":"300","volume-title":"Pac Symp Biocomput","author":"GR Lanckriet","year":"2004","unstructured":"Lanckriet GR, Deng M, Cristianini N, Jordan MI, Noble WS: Kernel-based data fusion and its application to protein function prediction in yeast. Pac Symp Biocomput. 2004, 300-311."},{"issue":"2","key":"570_CR4","doi-asserted-by":"publisher","first-page":"401","DOI":"10.1089\/10665270252935539","volume":"9","author":"P Pavlidis","year":"2002","unstructured":"Pavlidis P, Weston J, Cai J, Noble WS: Learning gene functional classifications from multiple data types. J Comput Biol. 2002, 9 (2): 401-411. 10.1089\/10665270252935539.","journal-title":"J Comput Biol"},{"issue":"Suppl 1","key":"570_CR5","doi-asserted-by":"publisher","first-page":"i38","DOI":"10.1093\/bioinformatics\/bti1016","volume":"21","author":"A Ben-Hur","year":"2005","unstructured":"Ben-Hur A, Noble WS: Kernel methods for predicting protein-protein interactions. Bioinformatics. 2005, 21 (Suppl 1): i38-i46. 10.1093\/bioinformatics\/bti1016.","journal-title":"Bioinformatics"},{"issue":"5644","key":"570_CR6","doi-asserted-by":"publisher","first-page":"449","DOI":"10.1126\/science.1087361","volume":"302","author":"R Jansen","year":"2003","unstructured":"Jansen R, Yu H, Greenbaum D, Kluger Y, Krogan NJ, Chung S, Emili A, Snyder M, Greenblatt JF, Gerstein M: A Bayesian networks approach for predicting protein-protein interactions from genomic data. Science. 2003, 302 (5644): 449-453. 10.1126\/science.1087361.","journal-title":"Science"},{"issue":"5701","key":"570_CR7","doi-asserted-by":"publisher","first-page":"1555","DOI":"10.1126\/science.1099511","volume":"306","author":"I Lee","year":"2004","unstructured":"Lee I, Date SV, Adai AT, Marcotte EM: A probabilistic functional network of yeast genes. Science. 2004, 306 (5701): 1555-1558. 10.1126\/science.1099511.","journal-title":"Science"},{"issue":"1","key":"570_CR8","doi-asserted-by":"publisher","first-page":"154","DOI":"10.1186\/1471-2105-5-154","volume":"5","author":"N Lin","year":"2004","unstructured":"Lin N, Wu B, Jansen R, Gerstein M, Zhao H: Information assessment on predicting protein-protein interactions. BMC Bioinformatics. 2004, 5 (1): 154-10.1186\/1471-2105-5-154.","journal-title":"BMC Bioinformatics"},{"issue":"14","key":"570_CR9","doi-asserted-by":"publisher","first-page":"8348","DOI":"10.1073\/pnas.0832373100","volume":"100","author":"OG Troyanskaya","year":"2003","unstructured":"Troyanskaya OG, Dolinski K, Owen AB, Altman RB, Botstein D: A Bayesian framework for combining heterogeneous data sources for gene function prediction (in Saccharomyces cerevisiae). Proc Natl Acad Sci USA. 2003, 100 (14): 8348-8353. 10.1073\/pnas.0832373100.","journal-title":"Proc Natl Acad Sci USA"},{"issue":"8","key":"570_CR10","doi-asserted-by":"publisher","first-page":"424","DOI":"10.1016\/j.tig.2005.06.006","volume":"21","author":"SL Wong","year":"2005","unstructured":"Wong SL, Zhang LV, Roth FP: Discovering functional relationships: biochemistry versus genetics. Trends Genet. 2005, 21 (8): 424-427. 10.1016\/j.tig.2005.06.006.","journal-title":"Trends Genet"},{"issue":"1","key":"570_CR11","doi-asserted-by":"publisher","first-page":"27","DOI":"10.1093\/nar\/28.1.27","volume":"28","author":"M Kanehisa","year":"2000","unstructured":"Kanehisa M, Goto S: KEGG: kyoto encyclopedia of genes and genomes. Nucleic Acids Res. 2000, 28 (1): 27-30. 10.1093\/nar\/28.1.27.","journal-title":"Nucleic Acids Res"},{"issue":"Suppl 1","key":"570_CR12","doi-asserted-by":"publisher","first-page":"I363","DOI":"10.1093\/bioinformatics\/bth910","volume":"20","author":"Y Yamanishi","year":"2004","unstructured":"Yamanishi Y, Vert JP, Kanehisa M: Protein network inference from multiple genomic data: a supervised approach. Bioinformatics. 2004, 20 (Suppl 1): I363-I370. 10.1093\/bioinformatics\/bth910.","journal-title":"Bioinformatics"},{"key":"570_CR13","unstructured":"GRIFn Home Page. [http:\/\/function.princeton.edu\/GRIFn]"},{"issue":"6868","key":"570_CR14","doi-asserted-by":"publisher","first-page":"141","DOI":"10.1038\/415141a","volume":"415","author":"AC Gavin","year":"2002","unstructured":"Gavin AC, Bosche M, Krause R, Grandi P, Marzioch M, Bauer A, Schultz J, Rick JM, Michon AM, Cruciat CM: Functional organization of the yeast proteome by systematic analysis of protein complexes. Nature. 2002, 415 (6868): 141-147. 10.1038\/415141a.","journal-title":"Nature"},{"issue":"5","key":"570_CR15","doi-asserted-by":"publisher","first-page":"919","DOI":"10.1016\/S0022-2836(03)00239-0","volume":"327","author":"E Sprinzak","year":"2003","unstructured":"Sprinzak E, Sattath S, Margalit H: How reliable are experimental protein-protein interaction data?. J Mol Biol. 2003, 327 (5): 919-923. 10.1016\/S0022-2836(03)00239-0.","journal-title":"J Mol Biol"},{"issue":"6770","key":"570_CR16","doi-asserted-by":"publisher","first-page":"623","DOI":"10.1038\/35001009","volume":"403","author":"P Uetz","year":"2000","unstructured":"Uetz P, Giot L, Cagney G, Mansfield TA, Judson RS, Knight JR, Lockshon D, Narayan V, Srinivasan M, Pochart P: A comprehensive analysis of protein-protein interactions in Saccharomyces cerevisiae. Nature. 2000, 403 (6770): 623-627. 10.1038\/35001009.","journal-title":"Nature"},{"issue":"6887","key":"570_CR17","doi-asserted-by":"publisher","first-page":"399","DOI":"10.1038\/nature750","volume":"417","author":"C von Mering","year":"2002","unstructured":"von Mering C, Krause R, Snel B, Cornell M, Oliver SG, Fields S, Bork P: Comparative assessment of large-scale data sets of protein-protein interactions. Nature. 2002, 417 (6887): 399-403. 10.1038\/nature750.","journal-title":"Nature"},{"issue":"3","key":"570_CR18","doi-asserted-by":"publisher","first-page":"381","DOI":"10.1093\/bioinformatics\/btg420","volume":"20","author":"SG Lee","year":"2004","unstructured":"Lee SG, Hur JU, Kim YS: A graph-theoretic modeling on GO space for biological interpretation of gene clusters. Bioinformatics. 2004, 20 (3): 381-388. 10.1093\/bioinformatics\/btg420.","journal-title":"Bioinformatics"},{"issue":"1","key":"570_CR19","doi-asserted-by":"publisher","first-page":"25","DOI":"10.1038\/75556","volume":"25","author":"M Ashburner","year":"2000","unstructured":"Ashburner M, Ball CA, Blake JA, Botstein D, Butler H, Cherry JM, Davis AP, Dolinski K, Dwight SS, Eppig JT: Gene ontology: tool for the unification of biology. The Gene Ontology Consortium. Nat Genet. 2000, 25 (1): 25-29. 10.1038\/75556.","journal-title":"Nat Genet"},{"issue":"1","key":"570_CR20","doi-asserted-by":"publisher","first-page":"77","DOI":"10.1093\/nar\/28.1.77","volume":"28","author":"CA Ball","year":"2000","unstructured":"Ball CA, Dolinski K, Dwight SS, Harris MA, Issel-Tarver L, Kasarskis A, Scafe CR, Sherlock G, Binkley G, Jin H: Integrating functional genomic information into the Saccharomyces genome database. Nucleic Acids Res. 2000, 28 (1): 77-80. 10.1093\/nar\/28.1.77.","journal-title":"Nucleic Acids Res"},{"issue":"1","key":"570_CR21","doi-asserted-by":"publisher","first-page":"31","DOI":"10.1093\/nar\/30.1.31","volume":"30","author":"HW Mewes","year":"2002","unstructured":"Mewes HW, Frishman D, Guldener U, Mannhaupt G, Mayer K, Mokrejs M, Morgenstern B, Munsterkotter M, Rudd S, Weil B: MIPS: a database for genomes and protein sequences. Nucleic Acids Res. 2002, 30 (1): 31-34. 10.1093\/nar\/30.1.31.","journal-title":"Nucleic Acids Res"},{"issue":"1","key":"570_CR22","doi-asserted-by":"publisher","first-page":"75","DOI":"10.1093\/nar\/29.1.75","volume":"29","author":"MC Costanzo","year":"2001","unstructured":"Costanzo MC, Crawford ME, Hirschman JE, Kranz JE, Olsen P, Robertson LS, Skrzypek MS, Braun BR, Hopkins KL, Kondu P: YPD, PombePD and WormPD: model organism volumes of the BioKnowledge library, an integrated resource for protein information. Nucleic Acids Res. 2001, 29 (1): 75-79. 10.1093\/nar\/29.1.75.","journal-title":"Nucleic Acids Res"},{"issue":"5","key":"570_CR23","doi-asserted-by":"publisher","first-page":"535","DOI":"10.1016\/j.mib.2004.08.012","volume":"7","author":"R Jansen","year":"2004","unstructured":"Jansen R, Gerstein M: Analyzing protein function on a genomic scale: the importance of gold-standard positives and negatives for network prediction. Curr Opin Microbiol. 2004, 7 (5): 535-545. 10.1016\/j.mib.2004.08.012.","journal-title":"Curr Opin Microbiol"},{"issue":"1","key":"570_CR24","doi-asserted-by":"publisher","first-page":"100","DOI":"10.1186\/1471-2105-6-100","volume":"6","author":"A Patil","year":"2005","unstructured":"Patil A, Nakamura H: Filtering high-throughput protein-protein interaction data using a combination of genomic features. BMC Bioinformatics. 2005, 6 (1): 100-10.1186\/1471-2105-6-100.","journal-title":"BMC Bioinformatics"},{"issue":"6959","key":"570_CR25","doi-asserted-by":"publisher","first-page":"686","DOI":"10.1038\/nature02026","volume":"425","author":"WK Huh","year":"2003","unstructured":"Huh WK, Falvo JV, Gerke LC, Carroll AS, Howson RW, Weissman JS, O''Shea EK: Global analysis of protein localization in budding yeast. Nature. 2003, 425 (6959): 686-691. 10.1038\/nature02026.","journal-title":"Nature"},{"issue":"5","key":"570_CR26","doi-asserted-by":"publisher","first-page":"349","DOI":"10.1074\/mcp.M100037-MCP200","volume":"1","author":"CM Deane","year":"2002","unstructured":"Deane CM, Salwinski L, Xenarios I, Eisenberg D: Protein interactions: two methods for assessment of the reliability of high throughput observations. Mol Cell Proteomics. 2002, 1 (5): 349-356. 10.1074\/mcp.M100037-MCP200.","journal-title":"Mol Cell Proteomics"},{"key":"570_CR27","first-page":"531","volume-title":"Pac Symp Biocomput","author":"Y Qi","year":"2005","unstructured":"Qi Y, Klein-Seetharaman J, Bar-Joseph Z: Random forest similarity for protein-protein interaction prediction from multiple sources. Pac Symp Biocomput. 2005, 531-542."},{"issue":"3","key":"570_CR28","doi-asserted-by":"publisher","first-page":"292","DOI":"10.1016\/j.sbi.2004.05.003","volume":"14","author":"P Bork","year":"2004","unstructured":"Bork P, Jensen LJ, von Mering C, Ramani AK, Lee I, Marcotte EM: Protein interaction networks from yeast to human. Curr Opin Struct Biol. 2004, 14 (3): 292-299. 10.1016\/j.sbi.2004.05.003.","journal-title":"Curr Opin Struct Biol"},{"issue":"1","key":"570_CR29","doi-asserted-by":"publisher","first-page":"289","DOI":"10.1093\/nar\/28.1.289","volume":"28","author":"I Xenarios","year":"2000","unstructured":"Xenarios I, Rice DW, Salwinski L, Baron MK, Marcotte EM, Eisenberg D: DIP: the database of interacting proteins. Nucleic Acids Res. 2000, 28 (1): 289-291. 10.1093\/nar\/28.1.289.","journal-title":"Nucleic Acids Res"},{"issue":"3","key":"570_CR30","doi-asserted-by":"publisher","first-page":"R23","DOI":"10.1186\/gb-2003-4-3-r23","volume":"4","author":"BJ Breitkreutz","year":"2003","unstructured":"Breitkreutz BJ, Stark C, Tyers M: The GRID: the General Repository for Interaction Datasets. Genome Biol. 2003, 4 (3): R23-10.1186\/gb-2003-4-3-r23.","journal-title":"Genome Biol"},{"key":"570_CR31","doi-asserted-by":"crossref","unstructured":"Alfarano C, Andrade CE, Anthony K, Bahroos N, Bajec M, Bantoft K, Betel D, Bobechko B, Boutilier K, Burgess E: The Biomolecular Interaction Network Database and related tools 2005 update. Nucleic Acids Res. 2005, D418-424. 33 Database","DOI":"10.1093\/nar\/gki051"},{"issue":"10","key":"570_CR32","doi-asserted-by":"publisher","first-page":"1275","DOI":"10.1093\/bioinformatics\/btg153","volume":"19","author":"PW Lord","year":"2003","unstructured":"Lord PW, Stevens RD, Brass A, Goble CA: Investigating semantic similarity measures across the Gene Ontology: the relationship between sequence and annotation. Bioinformatics. 2003, 19 (10): 1275-1283. 10.1093\/bioinformatics\/btg153.","journal-title":"Bioinformatics"},{"issue":"Suppl61","key":"570_CR33","first-page":"S2","volume":"7","author":"A Ben-Hur","year":"2005","unstructured":"Ben-Hur A, Noble WS: Choosing negative examples for the prediction of protein-protein interactions. BMC Bioinformatics. 2005, 7 (Suppl61): S2-","journal-title":"BMC Bioinformatics"},{"issue":"8","key":"570_CR34","doi-asserted-by":"publisher","first-page":"4569","DOI":"10.1073\/pnas.061034498","volume":"98","author":"T Ito","year":"2001","unstructured":"Ito T, Chiba T, Ozawa R, Yoshida M, Hattori M, Sakaki Y: A comprehensive two-hybrid analysis to explore the yeast protein interactome. Proc Natl Acad Sci U S A. 2001, 98 (8): 4569-4574. 10.1073\/pnas.061034498.","journal-title":"Proc Natl Acad Sci U S A"},{"issue":"5553","key":"570_CR35","doi-asserted-by":"publisher","first-page":"321","DOI":"10.1126\/science.1064987","volume":"295","author":"AH Tong","year":"2002","unstructured":"Tong AH, Drees B, Nardelli G, Bader GD, Brannetti B, Castagnoli L, Evangelista M, Ferracuti S, Nelson B, Paoluzi S: A combined experimental and computational strategy to define protein interaction networks for peptide recognition modules. Science. 2002, 295 (5553): 321-324. 10.1126\/science.1064987.","journal-title":"Science"},{"issue":"6868","key":"570_CR36","doi-asserted-by":"publisher","first-page":"180","DOI":"10.1038\/415180a","volume":"415","author":"Y Ho","year":"2002","unstructured":"Ho Y, Gruhler A, Heilbut A, Bader GD, Moore L, Adams SL, Millar A, Taylor P, Bennett K, Boutilier K: Systematic identification of protein complexes in Saccharomyces cerevisiae by mass spectrometry. Nature. 2002, 415 (6868): 180-183. 10.1038\/415180a.","journal-title":"Nature"},{"issue":"10","key":"570_CR37","doi-asserted-by":"publisher","first-page":"2987","DOI":"10.1091\/mbc.12.10.2987","volume":"12","author":"AP Gasch","year":"2001","unstructured":"Gasch AP, Huang M, Metzner S, Botstein D, Elledge SJ, Brown PO: Genomic expression responses to DNA-damaging agents and the regulatory role of the yeast ATR homolog Mec1p. Mol Biol Cell. 2001, 12 (10): 2987-3003.","journal-title":"Mol Biol Cell"},{"issue":"12","key":"570_CR38","doi-asserted-by":"publisher","first-page":"3273","DOI":"10.1091\/mbc.9.12.3273","volume":"9","author":"PT Spellman","year":"1998","unstructured":"Spellman PT, Sherlock G, Zhang MQ, Iyer VR, Anders K, Eisen MB, Brown PO, Botstein D, Futcher B: Comprehensive identification of cell cycle-regulated genes of the yeast Saccharomyces cerevisiae by microarray hybridization. Mol Biol Cell. 1998, 9 (12): 3273-3297.","journal-title":"Mol Biol Cell"}],"container-title":["BMC Genomics"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/link.springer.com\/content\/pdf\/10.1186\/1471-2164-7-187.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/link.springer.com\/article\/10.1186\/1471-2164-7-187\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/1471-2164-7-187.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,9,1]],"date-time":"2021-09-01T01:37:43Z","timestamp":1630460263000},"score":1,"resource":{"primary":{"URL":"https:\/\/bmcgenomics.biomedcentral.com\/articles\/10.1186\/1471-2164-7-187"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2006,7,25]]},"references-count":38,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2006,12]]}},"alternative-id":["570"],"URL":"http:\/\/dx.doi.org\/10.1186\/1471-2164-7-187","relation":{},"ISSN":["1471-2164"],"issn-type":[{"value":"1471-2164","type":"electronic"}],"subject":["Genetics","Biotechnology"],"published":{"date-parts":[[2006,7,25]]},"assertion":[{"value":"10 May 2006","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"25 July 2006","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"25 July 2006","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}}],"article-number":"187"}}' recorded_at: Sun, 12 Mar 2023 22:21:45 GMT - request: method: get uri: https://doi.org/ra/10.1186 body: encoding: UTF-8 string: '' headers: Connection: - close Host: - doi.org User-Agent: - http.rb/5.1.1 response: status: code: 200 message: OK headers: Date: - Sun, 12 Mar 2023 22:21:45 GMT Content-Type: - application/json;charset=UTF-8 Transfer-Encoding: - chunked Connection: - close Permissions-Policy: - interest-cohort=(),browsing-topics=() Cf-Cache-Status: - DYNAMIC Report-To: - '{"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v3?s=3SjAzRaW2PpRgm6d6BhirtdWAOBOa1STK%2F9cFroHnrPyM4%2Fv%2F2f9AXHlCLrZQoLtgBmmOHUGA8ulp0Bbmv%2FLGOa64n2eG%2FnnnDuVyu4q%2BVngj7%2FW1HboIJJvAJp4TCX6mBl0f9I%3D"}],"group":"cf-nel","max_age":604800}' Nel: - '{"success_fraction":0,"report_to":"cf-nel","max_age":604800}' Strict-Transport-Security: - max-age=31536000; includeSubDomains; preload Server: - cloudflare Cf-Ray: - 7a6f6e993e86b897-AMS Alt-Svc: - h3=":443"; ma=86400, h3-29=":443"; ma=86400 body: encoding: UTF-8 string: |- [ { "DOI": "10.1186", "RA": "Crossref" } ] recorded_at: Sun, 12 Mar 2023 22:21:45 GMT - request: method: get uri: https://api.crossref.org/works/10.1186/1471-2164-7-187/transform/application/vnd.crossref.unixsd+xml body: encoding: UTF-8 string: '' headers: Connection: - close Host: - api.crossref.org User-Agent: - http.rb/5.1.1 response: status: code: 200 message: OK headers: Date: - Sun, 12 Mar 2023 22:26:55 GMT Transfer-Encoding: - chunked Link: - ; rel="canonical", ; version="vor"; type="application/pdf"; rel="item", ; version="vor"; type="text/html"; rel="item", ; version="vor"; type="application/pdf"; rel="item", ; version="tdm"; rel="license" Access-Control-Expose-Headers: - Link Access-Control-Allow-Headers: - X-Requested-With, Accept, Accept-Encoding, Accept-Charset, Accept-Language, Accept-Ranges, Cache-Control Access-Control-Allow-Origin: - "*" Server: - Jetty(9.4.40.v20210413) X-Ratelimit-Limit: - '50' X-Ratelimit-Interval: - 1s X-Api-Pool: - public X-Rate-Limit-Limit: - '50' X-Rate-Limit-Interval: - 1s Permissions-Policy: - interest-cohort=() Connection: - close body: encoding: ASCII-8BIT string: "\n\r\n \r\n \ \r\n none\r\n \r\n \r\n \ \r\n 10.1186/1471-2164-7-187\r\n \ Springer Science and Business Media LLC\r\n Springer (Biomed Central Ltd.)\r\n 297\r\n 22586616\r\n 5911\r\n 20210831173149578\r\n 10.1186\r\n 2021-09-01T01:37:43Z\r\n 2006-07-25T18:14:51Z\r\n 171\r\n 10.3410/f.1044091.494000\r\n 10.3410/f.1044091.496329\r\n \ \r\n \r\n \ \r\n \r\n \ BMC Genomics\r\n BMC Genomics\r\n 1471-2164\r\n \ \r\n \r\n \r\n 12\r\n 2006\r\n \ \r\n \r\n \ 7\r\n \r\n \ 1\r\n \r\n \r\n \r\n Finding function: evaluation methods for functional genomic data\r\n \r\n \ \r\n \r\n Chad L\r\n \ Myers\r\n \r\n \ \r\n \ Daniel R\r\n Barrett\r\n \ \r\n \r\n Matthew A\r\n \ Hibbs\r\n \r\n \ \r\n \ Curtis\r\n Huttenhower\r\n \ \r\n \r\n Olga G\r\n \ Troyanskaya\r\n \r\n \ \r\n \r\n Abstract\r\n \ \r\n Background\r\n \ Accurate evaluation of the quality of genomic or proteomic data and computational methods is vital to our ability to use them for formulating novel biological hypotheses and directing further experiments. There is currently no standard approach to evaluation in functional genomics. Our analysis of existing approaches shows that they are inconsistent and contain substantial functional biases that render the resulting evaluations misleading both quantitatively and qualitatively. These problems make it essentially impossible to compare computational methods or large-scale experimental datasets and also result in conclusions that generalize poorly in most biological applications.\r\n \ \r\n \r\n Results\r\n \ We reveal issues with current evaluation methods here and suggest new approaches to evaluation that facilitate accurate and representative characterization of genomic methods and data. Specifically, we describe a functional genomics gold standard based on curation by expert biologists and demonstrate its use as an effective means of evaluation of genomic approaches. Our evaluation framework and gold standard are freely available to the community through our website.\r\n \r\n \ \r\n Conclusion\r\n \ Proper methods for evaluating genomic data and computational approaches will determine how much we, as a community, are able to learn from the wealth of available data. We propose one possible solution to this problem here but emphasize that this topic warrants broader community discussion.\r\n \r\n \r\n \ \r\n 07\r\n \ 25\r\n 2006\r\n \ \r\n \r\n \ 12\r\n 2006\r\n \ \r\n \r\n \ 187\r\n \ 570\r\n \r\n \ \r\n 1\r\n \ 10.1007/springer_crossmark_policy\r\n \ \r\n \r\n \ link.springer.com\r\n \r\n \ \r\n false\r\n \ \r\n 10 May 2006\r\n 25 July 2006\r\n 25 July 2006\r\n \r\n \ http://creativecommons.org/licenses/by/2.0\r\n \ \r\n \r\n \ \r\n \r\n 10.1186/1471-2164-7-187\r\n \ 20210831173149578\r\n https://bmcgenomics.biomedcentral.com/articles/10.1186/1471-2164-7-187\r\n \ \r\n \r\n https://link.springer.com/content/pdf/10.1186/1471-2164-7-187.pdf\r\n \ \r\n \r\n \r\n \r\n http://link.springer.com/content/pdf/10.1186/1471-2164-7-187.pdf\r\n \ \r\n \r\n http://link.springer.com/article/10.1186/1471-2164-7-187/fulltext.html\r\n \ \r\n \r\n \r\n \ \r\n \r\n \ Z Barutcuoglu\r\n 2006\r\n \ Bioinformatics\r\n Barutcuoglu Z, Schapire RE, Troyanskaya OG: Hierarchical multi-label prediction of gene function. Bioinformatics. 2006\r\n \r\n \ \r\n Bioinformatics\r\n \ A Clare\r\n 19\r\n \ Suppl 2\r\n II42\r\n \ 2003\r\n 10.1093/bioinformatics/btg1058\r\n \ Clare A, King RD: Predicting gene function in Saccharomyces cerevisiae. Bioinformatics. 2003, 19 (Suppl 2): II42-II49.\r\n \r\n \r\n GR Lanckriet\r\n \ 300\r\n 2004\r\n \ Pac Symp Biocomput\r\n Lanckriet GR, Deng M, Cristianini N, Jordan MI, Noble WS: Kernel-based data fusion and its application to protein function prediction in yeast. Pac Symp Biocomput. 2004, 300-311.\r\n \r\n \ \r\n J Comput Biol\r\n P Pavlidis\r\n \ 9\r\n 2\r\n \ 401\r\n 2002\r\n \ 10.1089/10665270252935539\r\n Pavlidis P, Weston J, Cai J, Noble WS: Learning gene functional classifications from multiple data types. J Comput Biol. 2002, 9 (2): 401-411. 10.1089/10665270252935539.\r\n \ \r\n \r\n \ Bioinformatics\r\n A Ben-Hur\r\n 21\r\n Suppl 1\r\n i38\r\n 2005\r\n \ 10.1093/bioinformatics/bti1016\r\n Ben-Hur A, Noble WS: Kernel methods for predicting protein-protein interactions. Bioinformatics. 2005, 21 (Suppl 1): i38-i46. 10.1093/bioinformatics/bti1016.\r\n \ \r\n \r\n \ Science\r\n R Jansen\r\n 302\r\n 5644\r\n \ 449\r\n 2003\r\n \ 10.1126/science.1087361\r\n Jansen R, Yu H, Greenbaum D, Kluger Y, Krogan NJ, Chung S, Emili A, Snyder M, Greenblatt JF, Gerstein M: A Bayesian networks approach for predicting protein-protein interactions from genomic data. Science. 2003, 302 (5644): 449-453. 10.1126/science.1087361.\r\n \ \r\n \r\n \ Science\r\n I Lee\r\n 306\r\n 5701\r\n \ 1555\r\n 2004\r\n \ 10.1126/science.1099511\r\n Lee I, Date SV, Adai AT, Marcotte EM: A probabilistic functional network of yeast genes. Science. 2004, 306 (5701): 1555-1558. 10.1126/science.1099511.\r\n \ \r\n \r\n \ BMC Bioinformatics\r\n \ N Lin\r\n 5\r\n \ 1\r\n 154\r\n \ 2004\r\n 10.1186/1471-2105-5-154\r\n \ Lin N, Wu B, Jansen R, Gerstein M, Zhao H: Information assessment on predicting protein-protein interactions. BMC Bioinformatics. 2004, 5 (1): 154-10.1186/1471-2105-5-154.\r\n \ \r\n \r\n \ Proc Natl Acad Sci USA\r\n \ OG Troyanskaya\r\n 100\r\n \ 14\r\n 8348\r\n \ 2003\r\n 10.1073/pnas.0832373100\r\n \ Troyanskaya OG, Dolinski K, Owen AB, Altman RB, Botstein D: A Bayesian framework for combining heterogeneous data sources for gene function prediction (in Saccharomyces cerevisiae). Proc Natl Acad Sci USA. 2003, 100 (14): 8348-8353. 10.1073/pnas.0832373100.\r\n \ \r\n \r\n \ Trends Genet\r\n SL Wong\r\n 21\r\n 8\r\n \ 424\r\n 2005\r\n \ 10.1016/j.tig.2005.06.006\r\n Wong SL, Zhang LV, Roth FP: Discovering functional relationships: biochemistry versus genetics. Trends Genet. 2005, 21 (8): 424-427. 10.1016/j.tig.2005.06.006.\r\n \ \r\n \r\n \ Nucleic Acids Res\r\n M Kanehisa\r\n 28\r\n 1\r\n \ 27\r\n 2000\r\n \ 10.1093/nar/28.1.27\r\n Kanehisa M, Goto S: KEGG: kyoto encyclopedia of genes and genomes. Nucleic Acids Res. 2000, 28 (1): 27-30. 10.1093/nar/28.1.27.\r\n \r\n \ \r\n Bioinformatics\r\n \ Y Yamanishi\r\n 20\r\n \ Suppl 1\r\n I363\r\n \ 2004\r\n 10.1093/bioinformatics/bth910\r\n \ Yamanishi Y, Vert JP, Kanehisa M: Protein network inference from multiple genomic data: a supervised approach. Bioinformatics. 2004, 20 (Suppl 1): I363-I370. 10.1093/bioinformatics/bth910.\r\n \ \r\n \r\n \ GRIFn Home Page. [http://function.princeton.edu/GRIFn]\r\n \ \r\n \r\n \ Nature\r\n AC Gavin\r\n 415\r\n 6868\r\n \ 141\r\n 2002\r\n \ 10.1038/415141a\r\n Gavin AC, Bosche M, Krause R, Grandi P, Marzioch M, Bauer A, Schultz J, Rick JM, Michon AM, Cruciat CM: Functional organization of the yeast proteome by systematic analysis of protein complexes. Nature. 2002, 415 (6868): 141-147. 10.1038/415141a.\r\n \ \r\n \r\n \ J Mol Biol\r\n E Sprinzak\r\n 327\r\n 5\r\n \ 919\r\n 2003\r\n \ 10.1016/S0022-2836(03)00239-0\r\n Sprinzak E, Sattath S, Margalit H: How reliable are experimental protein-protein interaction data?. J Mol Biol. 2003, 327 (5): 919-923. 10.1016/S0022-2836(03)00239-0.\r\n \ \r\n \r\n \ Nature\r\n P Uetz\r\n 403\r\n 6770\r\n \ 623\r\n 2000\r\n \ 10.1038/35001009\r\n Uetz P, Giot L, Cagney G, Mansfield TA, Judson RS, Knight JR, Lockshon D, Narayan V, Srinivasan M, Pochart P: A comprehensive analysis of protein-protein interactions in Saccharomyces cerevisiae. Nature. 2000, 403 (6770): 623-627. 10.1038/35001009.\r\n \ \r\n \r\n \ Nature\r\n C von Mering\r\n 417\r\n 6887\r\n \ 399\r\n 2002\r\n \ 10.1038/nature750\r\n von Mering C, Krause R, Snel B, Cornell M, Oliver SG, Fields S, Bork P: Comparative assessment of large-scale data sets of protein-protein interactions. Nature. 2002, 417 (6887): 399-403. 10.1038/nature750.\r\n \ \r\n \r\n \ Bioinformatics\r\n SG Lee\r\n 20\r\n 3\r\n \ 381\r\n 2004\r\n \ 10.1093/bioinformatics/btg420\r\n Lee SG, Hur JU, Kim YS: A graph-theoretic modeling on GO space for biological interpretation of gene clusters. Bioinformatics. 2004, 20 (3): 381-388. 10.1093/bioinformatics/btg420.\r\n \ \r\n \r\n \ Nat Genet\r\n M Ashburner\r\n 25\r\n 1\r\n \ 25\r\n 2000\r\n \ 10.1038/75556\r\n Ashburner M, Ball CA, Blake JA, Botstein D, Butler H, Cherry JM, Davis AP, Dolinski K, Dwight SS, Eppig JT: Gene ontology: tool for the unification of biology. The Gene Ontology Consortium. Nat Genet. 2000, 25 (1): 25-29. 10.1038/75556.\r\n \ \r\n \r\n \ Nucleic Acids Res\r\n CA Ball\r\n 28\r\n 1\r\n \ 77\r\n 2000\r\n \ 10.1093/nar/28.1.77\r\n Ball CA, Dolinski K, Dwight SS, Harris MA, Issel-Tarver L, Kasarskis A, Scafe CR, Sherlock G, Binkley G, Jin H: Integrating functional genomic information into the Saccharomyces genome database. Nucleic Acids Res. 2000, 28 (1): 77-80. 10.1093/nar/28.1.77.\r\n \r\n \ \r\n Nucleic Acids Res\r\n HW Mewes\r\n \ 30\r\n 1\r\n \ 31\r\n 2002\r\n \ 10.1093/nar/30.1.31\r\n Mewes HW, Frishman D, Guldener U, Mannhaupt G, Mayer K, Mokrejs M, Morgenstern B, Munsterkotter M, Rudd S, Weil B: MIPS: a database for genomes and protein sequences. Nucleic Acids Res. 2002, 30 (1): 31-34. 10.1093/nar/30.1.31.\r\n \ \r\n \r\n \ Nucleic Acids Res\r\n MC Costanzo\r\n 29\r\n 1\r\n \ 75\r\n 2001\r\n \ 10.1093/nar/29.1.75\r\n Costanzo MC, Crawford ME, Hirschman JE, Kranz JE, Olsen P, Robertson LS, Skrzypek MS, Braun BR, Hopkins KL, Kondu P: YPD, PombePD and WormPD: model organism volumes of the BioKnowledge library, an integrated resource for protein information. Nucleic Acids Res. 2001, 29 (1): 75-79. 10.1093/nar/29.1.75.\r\n \ \r\n \r\n \ Curr Opin Microbiol\r\n \ R Jansen\r\n 7\r\n \ 5\r\n 535\r\n \ 2004\r\n 10.1016/j.mib.2004.08.012\r\n \ Jansen R, Gerstein M: Analyzing protein function on a genomic scale: the importance of gold-standard positives and negatives for network prediction. Curr Opin Microbiol. 2004, 7 (5): 535-545. 10.1016/j.mib.2004.08.012.\r\n \r\n \ \r\n BMC Bioinformatics\r\n A Patil\r\n \ 6\r\n 1\r\n \ 100\r\n 2005\r\n \ 10.1186/1471-2105-6-100\r\n Patil A, Nakamura H: Filtering high-throughput protein-protein interaction data using a combination of genomic features. BMC Bioinformatics. 2005, 6 (1): 100-10.1186/1471-2105-6-100.\r\n \r\n \ \r\n Nature\r\n \ WK Huh\r\n 425\r\n \ 6959\r\n 686\r\n \ 2003\r\n 10.1038/nature02026\r\n \ Huh WK, Falvo JV, Gerke LC, Carroll AS, Howson RW, Weissman JS, O'Shea EK: Global analysis of protein localization in budding yeast. Nature. 2003, 425 (6959): 686-691. 10.1038/nature02026.\r\n \ \r\n \r\n \ Mol Cell Proteomics\r\n \ CM Deane\r\n 1\r\n \ 5\r\n 349\r\n \ 2002\r\n 10.1074/mcp.M100037-MCP200\r\n \ Deane CM, Salwinski L, Xenarios I, Eisenberg D: Protein interactions: two methods for assessment of the reliability of high throughput observations. Mol Cell Proteomics. 2002, 1 (5): 349-356. 10.1074/mcp.M100037-MCP200.\r\n \r\n \ \r\n Y Qi\r\n 531\r\n 2005\r\n \ Pac Symp Biocomput\r\n Qi Y, Klein-Seetharaman J, Bar-Joseph Z: Random forest similarity for protein-protein interaction prediction from multiple sources. Pac Symp Biocomput. 2005, 531-542.\r\n \ \r\n \r\n \ Curr Opin Struct Biol\r\n \ P Bork\r\n 14\r\n \ 3\r\n 292\r\n \ 2004\r\n 10.1016/j.sbi.2004.05.003\r\n \ Bork P, Jensen LJ, von Mering C, Ramani AK, Lee I, Marcotte EM: Protein interaction networks from yeast to human. Curr Opin Struct Biol. 2004, 14 (3): 292-299. 10.1016/j.sbi.2004.05.003.\r\n \ \r\n \r\n \ Nucleic Acids Res\r\n I Xenarios\r\n 28\r\n 1\r\n \ 289\r\n 2000\r\n \ 10.1093/nar/28.1.289\r\n Xenarios I, Rice DW, Salwinski L, Baron MK, Marcotte EM, Eisenberg D: DIP: the database of interacting proteins. Nucleic Acids Res. 2000, 28 (1): 289-291. 10.1093/nar/28.1.289.\r\n \ \r\n \r\n \ Genome Biol\r\n BJ Breitkreutz\r\n 4\r\n 3\r\n \ R23\r\n 2003\r\n \ 10.1186/gb-2003-4-3-r23\r\n Breitkreutz BJ, Stark C, Tyers M: The GRID: the General Repository for Interaction Datasets. Genome Biol. 2003, 4 (3): R23-10.1186/gb-2003-4-3-r23.\r\n \ \r\n \r\n \ 10.1093/nar/gki051\r\n \ Alfarano C, Andrade CE, Anthony K, Bahroos N, Bajec M, Bantoft K, Betel D, Bobechko B, Boutilier K, Burgess E: The Biomolecular Interaction Network Database and related tools 2005 update. Nucleic Acids Res. 2005, D418-424. 33 Database\r\n \ \r\n \r\n \ Bioinformatics\r\n PW Lord\r\n 19\r\n 10\r\n \ 1275\r\n 2003\r\n \ 10.1093/bioinformatics/btg153\r\n Lord PW, Stevens RD, Brass A, Goble CA: Investigating semantic similarity measures across the Gene Ontology: the relationship between sequence and annotation. Bioinformatics. 2003, 19 (10): 1275-1283. 10.1093/bioinformatics/btg153.\r\n \ \r\n \r\n \ BMC Bioinformatics\r\n \ A Ben-Hur\r\n 7\r\n \ Suppl61\r\n S2\r\n \ 2005\r\n Ben-Hur A, Noble WS: Choosing negative examples for the prediction of protein-protein interactions. BMC Bioinformatics. 2005, 7 (Suppl61): S2-\r\n \ \r\n \r\n \ Proc Natl Acad Sci U S A\r\n \ T Ito\r\n 98\r\n \ 8\r\n 4569\r\n \ 2001\r\n 10.1073/pnas.061034498\r\n \ Ito T, Chiba T, Ozawa R, Yoshida M, Hattori M, Sakaki Y: A comprehensive two-hybrid analysis to explore the yeast protein interactome. Proc Natl Acad Sci U S A. 2001, 98 (8): 4569-4574. 10.1073/pnas.061034498.\r\n \r\n \ \r\n Science\r\n \ AH Tong\r\n 295\r\n \ 5553\r\n 321\r\n \ 2002\r\n 10.1126/science.1064987\r\n \ Tong AH, Drees B, Nardelli G, Bader GD, Brannetti B, Castagnoli L, Evangelista M, Ferracuti S, Nelson B, Paoluzi S: A combined experimental and computational strategy to define protein interaction networks for peptide recognition modules. Science. 2002, 295 (5553): 321-324. 10.1126/science.1064987.\r\n \r\n \ \r\n Nature\r\n \ Y Ho\r\n 415\r\n \ 6868\r\n 180\r\n \ 2002\r\n 10.1038/415180a\r\n \ Ho Y, Gruhler A, Heilbut A, Bader GD, Moore L, Adams SL, Millar A, Taylor P, Bennett K, Boutilier K: Systematic identification of protein complexes in Saccharomyces cerevisiae by mass spectrometry. Nature. 2002, 415 (6868): 180-183. 10.1038/415180a.\r\n \ \r\n \r\n \ Mol Biol Cell\r\n AP Gasch\r\n 12\r\n 10\r\n \ 2987\r\n 2001\r\n \ 10.1091/mbc.12.10.2987\r\n Gasch AP, Huang M, Metzner S, Botstein D, Elledge SJ, Brown PO: Genomic expression responses to DNA-damaging agents and the regulatory role of the yeast ATR homolog Mec1p. Mol Biol Cell. 2001, 12 (10): 2987-3003.\r\n \ \r\n \r\n \ Mol Biol Cell\r\n PT Spellman\r\n 9\r\n 12\r\n \ 3273\r\n 1998\r\n \ 10.1091/mbc.9.12.3273\r\n Spellman PT, Sherlock G, Zhang MQ, Iyer VR, Anders K, Eisen MB, Brown PO, Botstein D, Futcher B: Comprehensive identification of cell cycle-regulated genes of the yeast Saccharomyces cerevisiae by microarray hybridization. Mol Biol Cell. 1998, 9 (12): 3273-3297.\r\n \r\n \ \r\n \r\n \r\n \ \r\n \r\n \r\n \r\n \ \r\n" recorded_at: Sun, 12 Mar 2023 22:26:55 GMT - request: method: get uri: https://api.crossref.org/members/297 body: encoding: UTF-8 string: '' headers: Connection: - close Host: - api.crossref.org User-Agent: - http.rb/5.1.1 response: status: code: 200 message: OK headers: Date: - Wed, 15 Mar 2023 08:40:03 GMT Content-Type: - application/json Transfer-Encoding: - chunked Access-Control-Expose-Headers: - Link Access-Control-Allow-Headers: - X-Requested-With, Accept, Accept-Encoding, Accept-Charset, Accept-Language, Accept-Ranges, Cache-Control Access-Control-Allow-Origin: - "*" Server: - Jetty(9.4.40.v20210413) X-Ratelimit-Limit: - '50' X-Ratelimit-Interval: - 1s X-Api-Pool: - public X-Rate-Limit-Limit: - '50' X-Rate-Limit-Interval: - 1s Permissions-Policy: - interest-cohort=() Connection: - close body: encoding: UTF-8 string: '{"status":"ok","message-type":"member","message-version":"1.0.0","message":{"last-status-check-time":1678836391304,"primary-name":"Springer Science and Business Media LLC","counts":{"current-dois":1598368,"backfile-dois":13838962,"total-dois":15437330},"breakdowns":{"dois-by-issued-year":[[2021,736358],[2022,733875],[2020,697212],[2019,637665],[2018,626548],[2017,607486],[2014,594641],[2016,587068],[2013,576881],[2015,524098],[2012,469435],[2011,453281],[2010,415699],[2009,396635],[2008,321987],[2007,283343],[2006,254428],[2004,251695],[2005,239229],[2003,235973],[2000,234800],[2002,224804],[2001,222693],[1999,205405],[1998,204586],[1997,198590],[1996,184966],[1995,174669],[1994,168573],[1993,166874],[1991,162690],[1992,161610],[1990,157196],[1989,152542],[1988,142665],[1987,136009],[2023,128119],[1986,125401],[1985,117346],[1984,114737],[1983,101538],[1982,99089],[1981,96286],[1980,91140],[1979,84510],[1978,83800],[1977,79820],[1976,73540],[1975,69728],[1974,67709],[1973,63097],[1972,60088],[1971,55889],[1970,53212],[1969,49982],[1968,49136],[1967,47329],[1966,42428],[1965,37392],[1964,33109],[1963,30507],[1962,30011],[1961,29323],[1960,28731],[1959,25568],[1958,24382],[1956,21824],[1957,21738],[1955,19848],[1954,16626],[1953,16613],[1932,16464],[1928,16360],[1930,16166],[1927,15856],[1926,15568],[1929,15312],[1931,15266],[1933,15121],[1952,14911],[1936,14695],[1938,14647],[1950,14397],[1935,14386],[1937,14277],[1939,14038],[1951,13848],[1934,13687],[1925,13399],[1924,13123],[1923,12807],[1922,12348],[1949,11829],[1913,11540],[1921,11423],[1940,11263],[1914,10095],[1941,9997],[1920,9920],[1942,9650],[1948,9414],[1912,9289],[1910,9125],[1911,9062],[1943,8570],[1904,8228],[1903,8049],[1909,7960],[1919,7948],[1908,7909],[1905,7551],[1907,7548],[1902,7371],[1906,7296],[1918,7183],[1901,7167],[1915,7030],[1900,7023],[1916,6836],[1897,6813],[1917,6752],[1899,6699],[1887,6682],[1886,6646],[1888,6500],[1947,6407],[1893,6403],[1898,6389],[1889,6349],[1895,6147],[1891,6106],[1880,6105],[1894,6100],[1944,6070],[1892,6066],[1885,6008],[1896,5963],[1890,5959],[1877,5912],[1884,5694],[1883,5675],[1878,5633],[1882,5623],[1881,5603],[1946,5556],[1876,5486],[1879,5427],[1847,5058],[1945,3954],[1848,3882],[1872,3827],[1875,3819],[1873,3761],[1870,3725],[1874,3641],[1871,3604],[1852,3064],[1849,3025],[1869,3021],[1856,2943],[1855,2836],[1857,2795],[1854,2772],[1851,2733],[1850,2690],[1860,2628],[1867,2586],[1859,2530],[1853,2512],[1868,2447],[1858,2430],[1865,2238],[1863,2148],[1864,2145],[1866,2110],[1862,2043],[1846,1919],[1861,1703],[1845,265],[1844,176],[1843,99],[1837,66],[1840,64],[1835,62],[1833,62],[1400,57],[1834,56],[1838,55],[1841,50],[1832,50],[1839,49],[1842,42],[1836,38],[2024,16],[1827,2],[1815,2]]},"prefixes":["10.1208","10.19150","10.1245","10.7603","10.1007","10.17269","10.4076","10.7123","10.1140","10.1381","10.33283","10.4333","10.1186","10.1065","10.1361","10.4056","10.1023","10.1385","10.26778","10.26777","10.5819","10.1617","10.1013","10.1057","10.1379","10.1038","10.1114","10.3758","10.1557","10.3858","10.7759","10.2165","10.1251","10.5052"],"coverage":{"affiliations-current":1.426454984083765E-4,"similarity-checking-current":0.9963788063825102,"descriptions-current":0.0,"ror-ids-current":0.0,"funders-backfile":0.04590250338139522,"licenses-backfile":0.7181505375908974,"funders-current":0.2503622444893792,"affiliations-backfile":4.790821739376118E-4,"resource-links-backfile":0.532235871447584,"orcids-backfile":0.05140327721110875,"update-policies-current":0.9308407075216721,"ror-ids-backfile":0.0,"orcids-current":0.3930465324631124,"similarity-checking-backfile":0.934759991392418,"references-backfile":0.6702399356252297,"descriptions-backfile":0.0,"award-numbers-backfile":0.03723256122821928,"update-policies-backfile":0.2792704395026159,"licenses-current":0.9950393151014034,"award-numbers-current":0.1939484524214699,"abstracts-backfile":0.03295283273413136,"resource-links-current":0.627209753949028,"abstracts-current":0.2149354841938778,"references-current":0.8618203067128472},"prefix":[{"name":"American Association of Pharmaceutical Scientists","value":"10.1208"},{"name":"Society for Mining, Metallurgy & Exploration Inc.","value":"10.19150"},{"name":"Springer - Society of Surgical Oncology","value":"10.1245"},{"name":"Springer - Global Science Journals","value":"10.7603"},{"name":"Springer-Verlag","value":"10.1007"},{"name":"Canadian Public Health Association","value":"10.17269"},{"name":"Springer (Cases Network, Ltd.)","value":"10.4076"},{"name":"Springer Science and Business Media LLC","value":"10.7123"},{"name":"Springer-Verlag","value":"10.1140"},{"name":"Springer - FD Communications","value":"10.1381"},{"name":"Springer Science and Business Media LLC","value":"10.33283"},{"name":"Springer - The Korean Society of Pharmaceutical Sciences and Technology","value":"10.4333"},{"name":"Springer (Biomed Central Ltd.)","value":"10.1186"},{"name":"Springer - Ecomed Publishers","value":"10.1065"},{"name":"Springer - ASM International","value":"10.1361"},{"name":"Genomic Standards Consortium","value":"10.4056"},{"name":"Springer (Kluwer Academic Publishers)","value":"10.1023"},{"name":"Springer - Humana Press","value":"10.1385"},{"name":"SpringerNature","value":"10.26778"},{"name":"SpringerNature","value":"10.26777"},{"name":"Springer - (backfiles)","value":"10.5819"},{"name":"Springer - RILEM Publishing","value":"10.1617"},{"name":"Springer Science and Business Media LLC","value":"10.1013"},{"name":"Springer Science and Business Media LLC","value":"10.1057"},{"name":"Springer - Cell Stress Society International","value":"10.1379"},{"name":"Springer Science and Business Media LLC","value":"10.1038"},{"name":"Springer (Kluwer Academic Publishers - Biomedical Engineering Society (BMES))","value":"10.1114"},{"name":"Springer - Psychonomic Society","value":"10.3758"},{"name":"Materials Research Society","value":"10.1557"},{"name":"Springer Science and Business Media LLC","value":"10.3858"},{"name":"Cureus, Inc.","value":"10.7759"},{"name":"Springer - Adis","value":"10.2165"},{"name":"Springer (Biological Procedures Online)","value":"10.1251"},{"name":"Springer - Real Academia de Ciencias Exactas, Fisicas y Naturales","value":"10.5052"}],"id":297,"tokens":["springer","science","and","business","media","llc"],"counts-type":{"all":{"monograph":359,"journal-article":9268581,"book":325977,"reference-entry":85798,"proceedings-article":1149,"journal":2149,"component":3720,"book-chapter":6354804,"reference-book":980,"journal-issue":1311,"dataset":23081},"backfile":{"monograph":359,"journal-article":8293758,"book":295579,"proceedings-article":1149,"book-chapter":5225953,"reference-book":744,"journal-issue":1311,"dataset":20109},"current":{"journal-article":974822,"book":30398,"book-chapter":589942,"reference-book":236,"dataset":2970}},"coverage-type":{"all":{"monograph":{"last-status-check-time":1678836391304,"affiliations":0.0,"abstracts":0.0,"orcids":0.0,"licenses":0.6295264623955432,"references":0.0,"funders":0.0,"similarity-checking":0.6295264623955432,"award-numbers":0.0,"ror-ids":0.0,"update-policies":0.0,"resource-links":0.6295264623955432,"descriptions":0.0},"journal-article":{"last-status-check-time":1678836391304,"affiliations":7.399190879380565E-4,"abstracts":0.08426823911880361,"orcids":0.1314145067082005,"licenses":0.8781779001553744,"references":0.7277159254474876,"funders":0.1098825159967853,"similarity-checking":0.9056900943089347,"award-numbers":0.08888048774672196,"ror-ids":0.0,"update-policies":0.3633847511285708,"resource-links":0.8677349855387788,"descriptions":0.0},"book":{"last-status-check-time":1678836391304,"affiliations":0.0,"abstracts":0.0,"orcids":0.02428698957288398,"licenses":0.9948738714694595,"references":3.067701095476061E-6,"funders":0.004165938087656491,"similarity-checking":0.996597919485117,"award-numbers":5.982017136178318E-4,"ror-ids":0.0,"update-policies":3.067701095476061E-6,"resource-links":0.9945149504412888,"descriptions":0.0},"reference-entry":{"last-status-check-time":1678836391304,"affiliations":0.0,"abstracts":0.0,"orcids":0.0,"licenses":0.0,"references":0.0,"funders":0.0,"similarity-checking":0.005396396186391291,"award-numbers":0.0,"ror-ids":0.0,"update-policies":0.0,"resource-links":0.0,"descriptions":0.0},"proceedings-article":{"last-status-check-time":1678836391304,"affiliations":0.0,"abstracts":0.0,"orcids":0.0,"licenses":0.0,"references":0.0,"funders":0.0,"similarity-checking":0.006092254134029591,"award-numbers":0.0,"ror-ids":0.0,"update-policies":0.0,"resource-links":0.0,"descriptions":0.0},"journal":{"last-status-check-time":1678836391304,"affiliations":0.0,"abstracts":0.0,"orcids":0.0,"licenses":4.653327128897161E-4,"references":0.0,"funders":0.0,"similarity-checking":0.001861330851558865,"award-numbers":0.0,"ror-ids":0.0,"update-policies":0.0,"resource-links":4.653327128897161E-4,"descriptions":0.0},"component":{"last-status-check-time":1678836391304,"affiliations":0.0,"abstracts":0.0,"orcids":0.0,"licenses":0.0,"references":0.0,"funders":0.0,"similarity-checking":0.0,"award-numbers":0.0,"ror-ids":0.0,"update-policies":0.0,"resource-links":0.0,"descriptions":8.064516129032258E-4},"book-chapter":{"last-status-check-time":1678836391304,"affiliations":0.0,"abstracts":0.00291621897386607,"orcids":0.01697943791814822,"licenses":0.4821391816332966,"references":0.6538064745978003,"funders":0.001342606317991869,"similarity-checking":0.9694931582468947,"award-numbers":1.995340847648488E-4,"ror-ids":0.0,"update-policies":0.3091165360882885,"resource-links":1.573612655874202E-7,"descriptions":0.0},"reference-book":{"last-status-check-time":1678836391304,"affiliations":0.0,"abstracts":0.0,"orcids":0.02040816326530612,"licenses":0.9908163265306122,"references":0.0,"funders":0.002040816326530612,"similarity-checking":0.9938775510204082,"award-numbers":0.002040816326530612,"ror-ids":0.0,"update-policies":0.0,"resource-links":0.9908163265306122,"descriptions":0.0},"journal-issue":{"last-status-check-time":1678836391304,"affiliations":0.0,"abstracts":0.0,"orcids":0.0,"licenses":0.03508771929824561,"references":0.0,"funders":0.0,"similarity-checking":0.07704042715484363,"award-numbers":0.0,"ror-ids":0.0,"update-policies":0.0,"resource-links":0.03508771929824561,"descriptions":0.0},"dataset":{"last-status-check-time":1678836391304,"affiliations":0.0,"abstracts":0.0,"orcids":0.2485594211689268,"licenses":0.0,"references":0.0,"funders":0.3061825744118539,"similarity-checking":0.0,"award-numbers":0.0,"ror-ids":0.0,"update-policies":0.8752653697846714,"resource-links":0.0,"descriptions":0.0}},"backfile":{"monograph":{"last-status-check-time":1678836391304,"affiliations":0.0,"abstracts":0.0,"orcids":0.0,"licenses":0.6295264623955432,"references":0.0,"funders":0.0,"similarity-checking":0.6295264623955432,"award-numbers":0.0,"ror-ids":0.0,"update-policies":0.0,"resource-links":0.6295264623955432,"descriptions":0.0},"journal-article":{"last-status-check-time":1678836391304,"affiliations":7.993963653147343E-4,"abstracts":0.05450038450603454,"orcids":0.08020332881668359,"licenses":0.8641989554071869,"references":0.7046928545539911,"funders":0.07525394399016706,"similarity-checking":0.8947485566856423,"award-numbers":0.0620168806468672,"ror-ids":0.0,"update-policies":0.2972635565204579,"resource-links":0.8525353645476514,"descriptions":0.0},"book":{"last-status-check-time":1678836391304,"affiliations":0.0,"abstracts":0.0,"orcids":0.01161449223388671,"licenses":0.9943500722311125,"references":3.383190280770961E-6,"funders":0.001918268889197135,"similarity-checking":0.9964882484885597,"award-numbers":3.586181697617219E-4,"ror-ids":0.0,"update-policies":3.383190280770961E-6,"resource-links":0.9941910622879163,"descriptions":0.0},"proceedings-article":{"last-status-check-time":1678836391304,"affiliations":0.0,"abstracts":0.0,"orcids":0.0,"licenses":0.0,"references":0.0,"funders":0.0,"similarity-checking":0.006092254134029591,"award-numbers":0.0,"ror-ids":0.0,"update-policies":0.0,"resource-links":0.0,"descriptions":0.0},"book-chapter":{"last-status-check-time":1678836391304,"affiliations":0.0,"abstracts":7.692376873653475E-4,"orcids":0.007471555905688398,"licenses":0.4738054475422952,"references":0.6565064783399315,"funders":9.860402495009044E-4,"similarity-checking":0.9987971571883635,"award-numbers":1.532734794974237E-4,"ror-ids":0.0,"update-policies":0.2639350181679782,"resource-links":1.913526585485939E-7,"descriptions":0.0},"reference-book":{"last-status-check-time":1678836391304,"affiliations":0.0,"abstracts":0.0,"orcids":0.01612903225806452,"licenses":0.9879032258064516,"references":0.0,"funders":0.0,"similarity-checking":0.9919354838709677,"award-numbers":0.0,"ror-ids":0.0,"update-policies":0.0,"resource-links":0.9879032258064516,"descriptions":0.0},"journal-issue":{"last-status-check-time":1678836391304,"affiliations":0.0,"abstracts":0.0,"orcids":0.0,"licenses":0.03508771929824561,"references":0.0,"funders":0.0,"similarity-checking":0.07704042715484363,"award-numbers":0.0,"ror-ids":0.0,"update-policies":0.0,"resource-links":0.03508771929824561,"descriptions":0.0},"dataset":{"last-status-check-time":1678836391304,"affiliations":0.0,"abstracts":0.0,"orcids":0.1834999254065344,"licenses":0.0,"references":0.0,"funders":0.2677905415485603,"similarity-checking":0.0,"award-numbers":0.0,"ror-ids":0.0,"update-policies":0.9979611119399274,"resource-links":0.0,"descriptions":0.0}},"current":{"journal-article":{"last-status-check-time":1678836391304,"affiliations":2.338888535548028E-4,"abstracts":0.3375323905287324,"orcids":0.5671178943437879,"licenses":0.9971102416646321,"references":0.9235963078387645,"funders":0.4045015397682859,"similarity-checking":0.9987802901452778,"award-numbers":0.3174353882042055,"ror-ids":0.0,"update-policies":0.9259423771724479,"resource-links":0.9970527952795485,"descriptions":0.0},"book":{"last-status-check-time":1678836391304,"affiliations":0.0,"abstracts":0.0,"orcids":0.147509704585828,"licenses":0.9999671030988881,"references":0.0,"funders":0.02602144877952497,"similarity-checking":0.997664320021054,"award-numbers":0.002927824198960458,"ror-ids":0.0,"update-policies":0.0,"resource-links":0.997664320021054,"descriptions":0.0},"book-chapter":{"last-status-check-time":1678836391304,"affiliations":0.0,"abstracts":0.02459902837906099,"orcids":0.1167148634950554,"licenses":0.9963708296747816,"references":0.8088320546765614,"funders":0.005727681704303135,"similarity-checking":0.9973590624163053,"award-numbers":7.916032423526381E-4,"ror-ids":0.0,"update-policies":0.9917296954615877,"resource-links":0.0,"descriptions":0.0},"reference-book":{"last-status-check-time":1678836391304,"affiliations":0.0,"abstracts":0.0,"orcids":0.03389830508474576,"licenses":1.0,"references":0.0,"funders":0.00847457627118644,"similarity-checking":1.0,"award-numbers":0.00847457627118644,"ror-ids":0.0,"update-policies":0.0,"resource-links":1.0,"descriptions":0.0},"dataset":{"last-status-check-time":1678836391304,"affiliations":0.0,"abstracts":0.0,"orcids":0.6892255892255892,"licenses":0.0,"references":0.0,"funders":0.5663299663299664,"similarity-checking":0.0,"award-numbers":0.0,"ror-ids":0.0,"update-policies":0.04511784511784512,"resource-links":0.0,"descriptions":0.0}}},"flags":{"deposits-abstracts-current":true,"deposits-orcids-current":true,"deposits":true,"deposits-affiliations-backfile":true,"deposits-update-policies-backfile":true,"deposits-similarity-checking-backfile":true,"deposits-award-numbers-current":true,"deposits-resource-links-current":true,"deposits-ror-ids-current":false,"deposits-articles":true,"deposits-affiliations-current":true,"deposits-funders-current":true,"deposits-references-backfile":true,"deposits-ror-ids-backfile":false,"deposits-abstracts-backfile":true,"deposits-licenses-backfile":true,"deposits-award-numbers-backfile":true,"deposits-descriptions-current":false,"deposits-references-current":true,"deposits-resource-links-backfile":true,"deposits-descriptions-backfile":false,"deposits-orcids-backfile":true,"deposits-funders-backfile":true,"deposits-update-policies-current":true,"deposits-similarity-checking-current":true,"deposits-licenses-current":true},"location":"Dordrecht, GX, Netherlands","names":["Springer Science and Business Media LLC","American Association of Pharmaceutical Scientists","Society for Mining, Metallurgy & Exploration Inc.","Springer - Society of Surgical Oncology","Springer - Global Science Journals","Springer-Verlag","Canadian Public Health Association","Springer (Cases Network, Ltd.)","Springer - FD Communications","Springer - The Korean Society of Pharmaceutical Sciences and Technology","Springer (Biomed Central Ltd.)","Springer - Ecomed Publishers","Springer - ASM International","Genomic Standards Consortium","Springer (Kluwer Academic Publishers)","Springer - Humana Press","SpringerNature","Springer - (backfiles)","Springer - RILEM Publishing","Springer - Cell Stress Society International","Springer (Kluwer Academic Publishers - Biomedical Engineering Society (BMES))","Springer - Psychonomic Society","Materials Research Society","Cureus, Inc.","Springer - Adis","Springer (Biological Procedures Online)","Springer - Real Academia de Ciencias Exactas, Fisicas y Naturales"]}}' recorded_at: Wed, 15 Mar 2023 08:40:03 GMT - request: method: get uri: https://doi.org/ra/10.1186 body: encoding: UTF-8 string: '' headers: Connection: - close Host: - doi.org User-Agent: - http.rb/5.1.1 response: status: code: 200 message: OK headers: Date: - Wed, 15 Mar 2023 09:09:32 GMT Content-Type: - application/json;charset=UTF-8 Transfer-Encoding: - chunked Connection: - close Permissions-Policy: - interest-cohort=(),browsing-topics=() Cf-Cache-Status: - DYNAMIC Report-To: - '{"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v3?s=MjQF6PykhhL0LjUbhrDjtkX5439T%2FjvdX9uEFMVAdsyDXizlbWhUrxVbb1MOP%2BWm1H57P0cx1lY23AViekDVUoOh182O%2BexcTgxomj8F%2BZUQM%2FRbeRxKSH%2Fmoz1GK4bsf%2Fy96IM%3D"}],"group":"cf-nel","max_age":604800}' Nel: - '{"success_fraction":0,"report_to":"cf-nel","max_age":604800}' Strict-Transport-Security: - max-age=31536000; includeSubDomains; preload Server: - cloudflare Cf-Ray: - 7a839e3dbd031c7a-AMS Alt-Svc: - h3=":443"; ma=86400, h3-29=":443"; ma=86400 body: encoding: UTF-8 string: |- [ { "DOI": "10.1186", "RA": "Crossref" } ] recorded_at: Wed, 15 Mar 2023 09:09:32 GMT recorded_with: VCR 6.1.0