SciPub course, Science, Community, Sciento- and Bibliometrics - - PowerPoint PPT Presentation

▶
scipub course science community sciento and bibliometrics
SMART_READER_LITE
LIVE PREVIEW

SciPub course, Science, Community, Sciento- and Bibliometrics - - PowerPoint PPT Presentation

Jun 01, 2023 108 likes •708 views

SciPub course, Science, Community, Sciento- and Bibliometrics 20121022, Chalmers, Gteborg Robert Feldt torsdag 25 oktober 12 torsdag 25 oktober 12 PhD course is not a Masters course! We are all colleagues We prepare and have discussions

slide-1
SLIDE 1

SciPub course, Science, Community, Sciento- and Bibliometrics

20121022, Chalmers, Göteborg Robert Feldt

torsdag 25 oktober 12

slide-2
SLIDE 2

torsdag 25 oktober 12

slide-3
SLIDE 3

PhD course is not a Master’s course!

We are all colleagues We prepare and have discussions Everyone active in discussions There is seldom one clear, correct answer I’ll give examples based on my experience, might not be the views of all faculty / your supervisor

torsdag 25 oktober 12

slide-4
SLIDE 4

torsdag 25 oktober 12

slide-5
SLIDE 5

What is science? Importance of community?

Lets timebox a discussion of 40 minutes based on your prep!

torsdag 25 oktober 12

slide-6
SLIDE 6

Definitions and background

Scientometrics is the science of measuring and analyzing science. Bibliometrics is the science of measuring and analyzing publications. Scientometrics is often done by applying bibliometrics on scientific publications. There are of course scientific journals in this research area

torsdag 25 oktober 12

slide-7
SLIDE 7

Example of papers in Scientometrics

http://www.springerlink.com/content/0138-9130 Bla: Claes Wohlin, “A new index for the citation curve of researchers”, Journal of Scientometrics, Vol 81, Num 2, November 2009. J

torsdag 25 oktober 12

slide-8
SLIDE 8

Recent papers in Scientometrics

“A citation-analysis of economic research institutes” “Impact factor: Imperfect but not yet replaceable” “Driving factors of external funding and funding effects on academic innovation performance in university–industry– government linkages” “Creative accomplishments in science: definition, theoretical considerations, examples from science history, and bibliometric findings”

torsdag 25 oktober 12

slide-9
SLIDE 9

Motivation (University perspective)

Several important reports during the last years: ”Bibliometriska undersökningen”, Akademi Sydost

Used a number of different metrics and sources

”Högskolerankingen 2007”, Sydsvenska Industri- och Handelskammaren

One criterion (out of 8) was the number of publications (indexed by the ISI Web of Science)

torsdag 25 oktober 12

slide-10
SLIDE 10

Motivation (University perspective)

”Resurser för kvalitet”, Utbildningsdepartementet

Suggests that governmental research funds should be based on the quality of the research, measured by e.g. the number of citations Forskningsproppen 2012 10-20% of total research funding should be awarded based

  • n “excellence”

Measured as both bibliometrics but also through expert assessment (international experts evaluate each major area)

torsdag 25 oktober 12

slide-11
SLIDE 11

Motivation (University perspective)

torsdag 25 oktober 12

slide-12
SLIDE 12

Motivation (University perspective)

VR wants everyone to state number of citations for every listed paper, but also says:

torsdag 25 oktober 12

slide-13
SLIDE 13

Motivation (University perspective)

VR wants everyone to state number of citations for every listed paper, but also says:

torsdag 25 oktober 12

slide-14
SLIDE 14

Motivation (University perspective)

VR wants everyone to state number of citations for every listed paper, but also says:

”Bibliometriska data, till exempel i form av det totala antalet citeringar för en sökande, bör användas med försiktighet i

  • beredningsarbetet. Det har visat sig att sådana kan bli

missvisande, om inte ett omfattande arbete med kvalitetssäkring utförts. Dock ska de sökande fr o m 2009 själva rapportera antalet citeringar för sina publikationer i ansökan.”

torsdag 25 oktober 12

slide-15
SLIDE 15

Better understanding of ”system”

How are researchers being evaluated/ranked? How are journals and conferences ranked?

Useful e.g. when

selecting where to publish your research results improving the impact of your research applying for positions applying for project funding

Motivation (individual perspective)

torsdag 25 oktober 12

slide-16
SLIDE 16

Motivation (individual perspective)

2011

torsdag 25 oktober 12

slide-17
SLIDE 17

Motivation (individual perspective)

2011

torsdag 25 oktober 12

slide-18
SLIDE 18

Motivation (individual perspective)

2011

torsdag 25 oktober 12

slide-19
SLIDE 19

Motivation (individual perspective)

2011

torsdag 25 oktober 12

slide-20
SLIDE 20

Even internal money at Chalmers will be distributed partly according to success in publishing in the “right” fora Unclear about GU but since our dept is both Chalmers and GU it will affect us

Motivation (individual perspective)

torsdag 25 oktober 12

slide-21
SLIDE 21

torsdag 25 oktober 12

slide-22
SLIDE 22

Metrics (scientists)

torsdag 25 oktober 12

slide-23
SLIDE 23

Metrics (scientists)

Total number of publications

torsdag 25 oktober 12

slide-24
SLIDE 24

Metrics (scientists)

Total number of publications Total number of citations

torsdag 25 oktober 12

slide-25
SLIDE 25

Metrics (scientists)

Total number of publications Total number of citations Average number of citations per paper

torsdag 25 oktober 12

slide-26
SLIDE 26

Metrics (scientists)

Total number of publications Total number of citations Average number of citations per paper H-index (proposed by Hirsch in 2005) An h-index of X means that a person have X publications that have X citations or more This metric is useful because it discounts the disproportionate weight of highly cited papers or papers that have not yet been cited. Combines quantity (number of publications) and quality (impact/ citations to these publications)

torsdag 25 oktober 12

slide-27
SLIDE 27

Metrics (scientists)

Total number of publications Total number of citations Average number of citations per paper H-index (proposed by Hirsch in 2005) An h-index of X means that a person have X publications that have X citations or more This metric is useful because it discounts the disproportionate weight of highly cited papers or papers that have not yet been cited. Combines quantity (number of publications) and quality (impact/ citations to these publications) The mean number of citations to offline articles is 2.74, whereas the mean number of citations to online articles is 7.03!

torsdag 25 oktober 12

slide-28
SLIDE 28

Metrics (scientists)

Total number of publications Total number of citations Average number of citations per paper H-index (proposed by Hirsch in 2005) An h-index of X means that a person have X publications that have X citations or more This metric is useful because it discounts the disproportionate weight of highly cited papers or papers that have not yet been cited. Combines quantity (number of publications) and quality (impact/ citations to these publications) The mean number of citations to offline articles is 2.74, whereas the mean number of citations to online articles is 7.03! (Lawrence, 2001)

torsdag 25 oktober 12

slide-29
SLIDE 29

Main sources of bibliometric data

torsdag 25 oktober 12

slide-30
SLIDE 30

Main sources of bibliometric data

ISI Web of Science (WoS) (http://portal.isiknowledge.com/)

  • nly (a selection of) journals (about 9000)

favors areas like the medical and natural sciences at the expense of areas where conference publications are important, like computer science as well as, other areas which mainly publish books and/or reports, like the humanities

torsdag 25 oktober 12

slide-31
SLIDE 31

Main sources of bibliometric data

ISI Web of Science (WoS) (http://portal.isiknowledge.com/)

  • nly (a selection of) journals (about 9000)

favors areas like the medical and natural sciences at the expense of areas where conference publications are important, like computer science as well as, other areas which mainly publish books and/or reports, like the humanities Google Scholar (GS) (http://scholar.google.se/) includes ”everything” available on the www but have problems with accuracy nice software tool available called ”Publish or perish” (http:// www.harzing.com/), but increasingly included in GS directly...

torsdag 25 oktober 12

slide-32
SLIDE 32

Metric for publications: Impact Factor

torsdag 25 oktober 12

slide-33
SLIDE 33

Metric for publications: Impact Factor

The average number of cites to articles in a journal/proceedings

torsdag 25 oktober 12

slide-34
SLIDE 34

Metric for publications: Impact Factor

The average number of cites to articles in a journal/proceedings PageRank (similar to Google’s algorithm) is a recursive impact factor, to give citations from journals that have high impact greater weight than citations from low-impact journals

torsdag 25 oktober 12

slide-35
SLIDE 35

Metric for publications: Impact Factor

The average number of cites to articles in a journal/proceedings PageRank (similar to Google’s algorithm) is a recursive impact factor, to give citations from journals that have high impact greater weight than citations from low-impact journals Objective and info widely available

torsdag 25 oktober 12

slide-36
SLIDE 36

Metric for publications: Impact Factor

The average number of cites to articles in a journal/proceedings PageRank (similar to Google’s algorithm) is a recursive impact factor, to give citations from journals that have high impact greater weight than citations from low-impact journals Objective and info widely available The number of citations to papers in a particular journal does not really directly measure the true quality of a journal, much less the scientific merit of the papers within it.

torsdag 25 oktober 12

slide-37
SLIDE 37

Metric for publications: Impact Factor

The average number of cites to articles in a journal/proceedings PageRank (similar to Google’s algorithm) is a recursive impact factor, to give citations from journals that have high impact greater weight than citations from low-impact journals Objective and info widely available The number of citations to papers in a particular journal does not really directly measure the true quality of a journal, much less the scientific merit of the papers within it. Cannot be compared directly between different research areas/

  • disciplines. It depends on, e.g.:

the absolute number of researchers in the area, the average number of authors on each paper, the nature of results in different research areas, citation habits between different disciplines, particularly the number of citations in each paper

torsdag 25 oktober 12

slide-38
SLIDE 38

Metric for publications: Impact Factor

The average number of cites to articles in a journal/proceedings PageRank (similar to Google’s algorithm) is a recursive impact factor, to give citations from journals that have high impact greater weight than citations from low-impact journals Objective and info widely available The number of citations to papers in a particular journal does not really directly measure the true quality of a journal, much less the scientific merit of the papers within it. Cannot be compared directly between different research areas/

  • disciplines. It depends on, e.g.:

the absolute number of researchers in the area, the average number of authors on each paper, the nature of results in different research areas, citation habits between different disciplines, particularly the number of citations in each paper Review/survey journals always get a lot of citations

torsdag 25 oktober 12

slide-39
SLIDE 39

Sources (Publications)

torsdag 25 oktober 12

slide-40
SLIDE 40

Sources (Publications)

Impact Factor ISI Journal Citation Reports (http://portal.isiknowledge.com/) CiteSeer (http://citeseer.ist.psu.edu/impact.html) Libra Academic Search (http://libra.msra.cn/) www.eigenfactor.org (a PageRank-type measure)

torsdag 25 oktober 12

slide-41
SLIDE 41

Sources (Publications)

Impact Factor ISI Journal Citation Reports (http://portal.isiknowledge.com/) CiteSeer (http://citeseer.ist.psu.edu/impact.html) Libra Academic Search (http://libra.msra.cn/) www.eigenfactor.org (a PageRank-type measure) Acceptance rate www.cs.ucsb.edu/~almeroth/conf/stats/

torsdag 25 oktober 12

slide-42
SLIDE 42

Sources (Publications)

Impact Factor ISI Journal Citation Reports (http://portal.isiknowledge.com/) CiteSeer (http://citeseer.ist.psu.edu/impact.html) Libra Academic Search (http://libra.msra.cn/) www.eigenfactor.org (a PageRank-type measure) Acceptance rate www.cs.ucsb.edu/~almeroth/conf/stats/ Ranking / Prestige (not really metrics) often unofficial lists, e.g www.ntu.edu.sg/home/assourav/crank.htm www.cs.ualberta.ca/~zaiane/htmldocs/ConfRanking.html Journal Quality List (http://www.harzing.com/)

torsdag 25 oktober 12

slide-43
SLIDE 43

Sources for SE

torsdag 25 oktober 12

slide-44
SLIDE 44

Sources for SE

Tao Xie’s ”Software Eng. Conferences” and ”SE Conf Map” http://people.engr.ncsu.edu/txie/seconferences.htm http://research.csc.ncsu.edu/ase/semap

torsdag 25 oktober 12

slide-45
SLIDE 45

Sources for SE

Tao Xie’s ”Software Eng. Conferences” and ”SE Conf Map” http://people.engr.ncsu.edu/txie/seconferences.htm http://research.csc.ncsu.edu/ase/semap Australian Ranking of ICT Conferences (2007) http://www.core.edu.au/rankings/Conference%20Ranking %20Main.html

http://www.arc.gov.au/era/tiers_ranking.htm http://core.edu.au/cms/images/downloads/ conference/Astar.pdf

http://www.arc.gov.au/era/era_2012/era_journal_list.htm

torsdag 25 oktober 12

slide-46
SLIDE 46

Sources for SE

torsdag 25 oktober 12

slide-47
SLIDE 47

Sources for SE

Journal of Systems and Software (JSS) yearly ranking Glass et al since 1996, 2011 one covers 2004-2008 1.0 points if 1 au, 0.7/au for 2 au, 0.5/au for 3, 0.4/au for >3 Based on only 7 journals (TSE, TOSEM, JSS, SPE, EMSE, IST, IEEE SW) Wohlin, Runeson, Bosch only “Swe” individuals, BTH only group, Chalmers/GU has goal to be top-5 in 5 years

torsdag 25 oktober 12

slide-48
SLIDE 48

Sources for SE

Journal of Systems and Software (JSS) yearly ranking Glass et al since 1996, 2011 one covers 2004-2008 1.0 points if 1 au, 0.7/au for 2 au, 0.5/au for 3, 0.4/au for >3 Based on only 7 journals (TSE, TOSEM, JSS, SPE, EMSE, IST, IEEE SW) Wohlin, Runeson, Bosch only “Swe” individuals, BTH only group, Chalmers/GU has goal to be top-5 in 5 years Feldt has summary lists: http://www.cse.chalmers.se/~feldt/advice/ isi_listed_se_journals.html

torsdag 25 oktober 12

slide-49
SLIDE 49

torsdag 25 oktober 12

slide-50
SLIDE 50

torsdag 25 oktober 12

slide-51
SLIDE 51

torsdag 25 oktober 12

slide-52
SLIDE 52

Questions to discuss

torsdag 25 oktober 12

slide-53
SLIDE 53

Questions to discuss

a, What are the most common bibliometric measures out there and how do they work?

torsdag 25 oktober 12

slide-54
SLIDE 54

Questions to discuss

a, What are the most common bibliometric measures out there and how do they work?

torsdag 25 oktober 12

slide-55
SLIDE 55

Questions to discuss

a, What are the most common bibliometric measures out there and how do they work? b, What are the disadvantages/advantages of using such measures?

torsdag 25 oktober 12

slide-56
SLIDE 56

Questions to discuss

a, What are the most common bibliometric measures out there and how do they work? b, What are the disadvantages/advantages of using such measures?

torsdag 25 oktober 12

slide-57
SLIDE 57

Questions to discuss

a, What are the most common bibliometric measures out there and how do they work? b, What are the disadvantages/advantages of using such measures? c, Does the usefulness of the measures differ between research areas? For example, is the medical area (as per the intro text from KI) different from Software Engineering?

torsdag 25 oktober 12

slide-58
SLIDE 58

Questions to discuss

a, What are the most common bibliometric measures out there and how do they work? b, What are the disadvantages/advantages of using such measures? c, Does the usefulness of the measures differ between research areas? For example, is the medical area (as per the intro text from KI) different from Software Engineering?

torsdag 25 oktober 12

slide-59
SLIDE 59

Questions to discuss

a, What are the most common bibliometric measures out there and how do they work? b, What are the disadvantages/advantages of using such measures? c, Does the usefulness of the measures differ between research areas? For example, is the medical area (as per the intro text from KI) different from Software Engineering? d, Should I be aware of these measures in my own career/project? How?

torsdag 25 oktober 12