COMMunication
RESearch.NET

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
====== References ======
  * http://www.rdatamining.com/docs/r-and-data-mining-examples-and-case-studies
    * book download {{:RDataMining-book.pdf|R and Data Mining}}

  * http://www.openhangul.com/
  * http://statmath.wu.ac.at/courses/SNLP/Presentations/DA-Sentiment.pdf
  * http://word.snu.ac.kr/kosac/
  * http://mpqa.cs.pitt.edu/lexicons/subj_lexicon/

  * {{https://pdfs.semanticscholar.org/07f3/d4ee8ade9f3802fb89720d68c56c945ff453.pdf}}

  * http://rconference.fossa.kr/handout/sentiment_analysis_hyungjunkim.pdf
  * http://blog.naver.com/PostList.nhn?blogId=wcjpower

Introduction 
Introduction to [[:social network analysis]]
  * [[amazon>0521387078]]

data file: {{:textmining.zip}} 
create Textmining directory in R working directory.
Unzip the zip file. 
====== e.g. 1 ======
<code>
NeededPackages <- c("tm", "SnowballC", "RColorBrewer", "ggplot2", "wordcloud", "biclust", 
    "cluster", "igraph", "fpc")
install.packages(NeededPackages, dependencies = TRUE)
</code>

<code>Sys.setlocale(category = "LC_ALL", locale = "US")
</code>

<code>library(tm)
</code>

<code>#Create Corpus
docs <- Corpus(DirSource("C:/Users/Hyo/Documents/R/textMining"))
docs
</code>

{{:pasted:20220614-102341.png?650}}

<code>#inspect a particular document
writeLines(as.character(docs[[30]]))
</code>

<code>getTransformations()

</code>

<code>
#create the toSpace content transformer
toSpace <- content_transformer(function(x, pattern) {return (gsub(pattern, " ", x))})
</code>

<code>
docs <- tm_map(docs, toSpace, "-")
docs <- tm_map(docs, toSpace, ":")
</code>

<code>#Remove punctuation ? replace punctuation marks with " "
docs <- tm_map(docs, removePunctuation)

docs <- tm_map(docs, toSpace, "’")
docs <- tm_map(docs, toSpace, "‘")
docs <- tm_map(docs, toSpace, " -")
</code>

<code>#Transform to lower case (need to wrap in content_transformer)
docs <- tm_map(docs,content_transformer(tolower))
</code>

<code>#Strip digits (std transformation, so no need for content_transformer)
docs <- tm_map(docs, removeNumbers)
</code>

<code>
#remove stopwords using the standard list in tm
docs <- tm_map(docs, removeWords, stopwords("english"))
</code>

<code>
#Strip whitespace (cosmetic?)
docs <- tm_map(docs, stripWhitespace)
</code>

<code>
writeLines(as.character(docs[[30]]))
</code>

<code>
#load library
library(SnowballC)

#Stem document
docs <- tm_map(docs,stemDocument)
writeLines(as.character(docs[[30]]))
</code>

<code>
docs <- tm_map(docs, content_transformer(gsub), pattern = "organiz", replacement = "organ")
docs <- tm_map(docs, content_transformer(gsub), pattern = "organis", replacement = "organ")
docs <- tm_map(docs, content_transformer(gsub), pattern = "andgovern", replacement = "govern")
docs <- tm_map(docs, content_transformer(gsub), pattern = "inenterpris", replacement = "enterpris")
docs <- tm_map(docs, content_transformer(gsub), pattern = "team-", replacement = "team")
</code>

<code>dtm <- DocumentTermMatrix(docs)
dtm
</code>

<code>
inspect(dtm[1:2,1000:1005])
</code>

<code>freq <- colSums(as.matrix(dtm))
</code>

<code>#length should be total number of terms
length(freq)
</code>

<code>
#create sort order (descending)
ord <- order(freq,decreasing=TRUE)
</code>

<code>
#inspect most frequently occurring terms
freq[head(ord)]

#inspect least frequently occurring terms
freq[tail(ord)]   
</code>

<code>
dtmr <-DocumentTermMatrix(docs, control=list(wordLengths=c(4, 20), bounds = list(global = c(3,27))))
</code>

<code>dtmr
</code>

<code>
freqr <- colSums(as.matrix(dtmr))
#length should be total number of terms
length(freqr)

#create sort order (asc)
ordr <- order(freqr,decreasing=TRUE)

#inspect most frequently occurring terms
freqr[head(ordr)]

#inspect least frequently occurring terms
freqr[tail(ordr)]
</code>

<code>
findFreqTerms(dtmr,lowfreq=80)
</code>

<code>
findAssocs(dtmr, "project", 0.6)
findAssocs(dtmr, "enterpris", 0.6)
findAssocs(dtmr, "system", 0.6)
</code>

<code>
wf=data.frame(term=names(freqr),occurrences=freqr)
library(ggplot2)
p <- ggplot(subset(wf, freqr>100), aes(term, occurrences))
p <- p + geom_bar(stat="identity")
p <- p + theme(axis.text.x=element_text(angle=45, hjust=1))
p
</code>

<code>#wordcloud
library(wordcloud)
#setting the same seed each time ensures consistent look across clouds
set.seed(42)
#limit words by specifying min frequency
wordcloud(names(freqr),freqr, min.freq=70)
</code>

<code>
#…add color
wordcloud(names(freqr),freqr,min.freq=70,colors=brewer.pal(6,"Dark2"))

</code>

====== e.g. 1 with output ======

<code>> Sys.setlocale(category = "LC_ALL", locale = "US")
[1] "LC_COLLATE=English_United States.1252;LC_CTYPE=English_United States.1252;LC_MONETARY=English_United States.1252;LC_NUMERIC=C;LC_TIME=English_United States.1252"
> library(tm)
> #Create Corpus
> docs <- Corpus(DirSource("D:/Users/Hyo/Documents/TextMining"))
> docs
<<SimpleCorpus>>
Metadata:  corpus specific: 1, document level (indexed): 0
Content:  documents: 30
> #inspect a particular document
> writeLines(as.character(docs[[30]]))
Understanding "flexibility" ?a close-up view of an organizational platitude

Introduction
Flexibility is one of those buzzwords that keeps coming up in organizational communiques and discussions. People are continually asked to display flexibility, without ever being told what the term means:  flexible workplaces, flexible attitudes, flexible jobs ?the word itself has a flexible meaning that depends on the context in which it is used and by whom.
When words are used in this way they become platitudes ?empty words that make a lot of noise. In this post, I analyse the platitude, flexibility, as it is used in organisations. My discussion is based on a paper by Thomas Eriksen entitled, Mind the Gap: Flexibility, Epistemology and the Rhetoric of New Work.
Background ?a bit about organizational platitudes
One of the things that struck me when I moved from academia to industry is the difference in the way words or phrases are used in the two domains. In academics one has to carefully define the terms one uses (particularly if one is coining a new term) whereas in business it doesn't seem to matter, words can mean whatever one wants them to mean (OK, this is an exaggeration, but not by too much). Indeed, as Paul Culmsee and I discuss in the first chapter of The Heretic's Guide to Best Practices, many terms that are commonly bandied about in organizations are platitudes because they are understood differently by different people.
A good example of a platitude is the word governance. One manager may see governance as being largely about oversight and control whereas another might interpret it as being about providing guidance.  Such varying interpretations can result in major differences in the way the two managers implement governance:  the first one might enforce it as a compliance-oriented set of processes that leave little room for individual judgement while the other might implement it as a broad set of guidelines that leave many of the decisions in the hands of those who are actually doing the work. Needless to say, the results in the two cases are likely to be different too.
Flexibility ?the conventional view
A good place to start our discussion of flexibility is with the dictionary. The online Oxford Dictionary defines at as:
Flexibility (noun):
 the ability to be easily modified
 willingness to change or compromise
The term is widely used in both these senses in organizational settings. For example, people speak of flexible designs (i.e. designs that can be easily modified) or flexible people (referring to those who are willing to change or compromise). However,  and this is the problem:  the term is open to interpretation ?what Jack might term a flexible approach may be seen by Jill as a complete lack of method. These differences in interpretation become particularly obvious when the word is used in a broad context ?such as in a statement justifying an organizational change.  An executive might see a corporate restructure and the resulting changes in jobs/roles as a means to achieve organizational flexibility, but those affected by it may see it as constraining theirs.  As Eriksen states:
Jobs are flexible in the sense that they are unstable and uncertain, few employees hold the same jobs for many years, the content of jobs can be changed almost overnight, and the boundaries between work and leisure are negotiable and chronically fuzzy.
Indeed, such "flexibility" which requires one to change at short notice results in a fragmentation of individual experience and a resulting loss of a coherent narrative of one's life. It appears that increased flexibility in one aspect results in a loss of flexibility in another. Any sensible definition of flexibility ought to reflect this.
Understanding flexibility
Consider the following definition of flexibility proposed by Gregory Bateson:
"Flexibility is uncommitted potential for change"
This deceptively simple statement is a good place to start understanding what flexibility really means for projects, organisations 꿢nd even software systems.
As Eriksen tells us, Bateson proposed this definition in the context of ecology. In particular, Bateson had in mind the now obvious notion that the increased flexibility we gain through our increasingly energy-hungry lifestyles results in a decrease in the environment's capacity to cope with the consequences. This is true of flexibility in any context: a gain in flexibility in one dimension will necessarily be accompanied by a loss of flexibility in another.
Another implication of the above definition is that a system that is running at or near the limits of its operating variables cannot be flexible.  The following examples should make this clear:
 A project team that is putting in 18 hour workdays in order to finish a project on time.
 A car that's being driven at top speed.
 A family living beyond their means.
All these systems are operating at or near their limits, they have little or no spare capacity to accommodate change.
A third implication of the definition follows from the preceding one:  the key variables of a flexible system should lie in the mid-range of their upper and lower limits. In terms of above examples:
 The project team should be putting in normal hours.
 The car should be driven at or below the posted road speed limits
 The family should be living within its income, with a reasonable amount to spare.
Of course, the whole point of ensuring that systems operate in their comfort zone is that they can be revved up if the need arises. Such revving up, however,  should be an exceptional circumstance rather than the norm ?a point that those who run projects, organisations (and, yes, even vehicles) often tend to forget. If one operates a system at the limits of its tolerance for too long, not only will it not be flexible, it will break.
Flexibility in the workplace
As mentioned in the introduction, the term flexibility keeps cropping up in organizational settings: corporate communiques exhort employees to be flexible in the face of change.  This is typically a coded signal that employees should expect uncertainty and be prepared to adjust to it.  A related manifestation of flexibility is the blurring of the distinction between work and personal life. As Eriksen puts it:
The term flexibility is often used to describe this new situation: Jobs are flexible in the sense that they are unstable and uncertain, few employees hold the same jobs for many years, the content of jobs can be changed, and the boundaries between work and leisure are poorly defined.
This trend is aided by recent developments in technology that enable employees to be perpetually on call. This is often sold as a work from home initiative but usually ends up being much more.  Eriksen has this to say about home offices:
One recent innovation typically associated with flexibility is the home office. In Scandinavia (and some other prosperous, technologically optimistic regions), many companies equipped some of their employees with home computers with online access to the company network in the early 1990s, in order to enhance their flexibility. This was intended to enable employees to work from home part of the time, thereby making the era when office workers were chained to the office desk all day obsolete.
In the early days, there were widespread worries among employers to the effect that a main outcome of this new flexibility would consist in a reduction of productivity. Since there was no legitimate way of checking how the staff actually spent their time out of the office, it was often suspected that they worked less from home than they were supposed to. If this were in fact the case, working from home would have led to a real increase in the flexibility of time budgeting. However, work researchers eventually came up with a different picture. By the late 1990s, hardly anybody spoke of the home office as a convenient way of escaping from work; rather, the concern among unionists as well as researchers was now that increasing numbers of employees were at pains to distinguish between working hours and leisure time, and were suffering symptoms of burnout and depression. The home office made it difficult to distinguish between contexts that were formerly mutually exclusive because of differ... <truncated>
It is interesting to see this development in the light of Bateson's definition of flexibility: the employee gains flexibility in space (he or she can work from home or from the office) at the expense of flexibility in time(organization time encroaches on personal time). As Eriksen states:
 There seems to be a classic Batesonian flexibility trade-off associated with the new information technologies: increased spatial flexibility entails decreased temporal flexibility. If inaccessibility and 'empty time' are understood as scarce resources, the context of 'new work' thus seems to be an appropriate context for a new economics as well. In fact, a main environmental challenge of our near future will consist in protecting slow time and gaps from environmental degradation.
In short, it appears that flexibility for the organization necessarily implies a loss of flexibility for the individual.
Conclusion
Flexibility is in the eye of the beholder: an action to increase organisational flexibility by, say, redeploying employees would likely be seen by those affected as a move that constrains their  (individual) flexibility.  Such a dual meaning is characteristic of many organizational platitudes such as Excellence, Synergy andGovernance. It is an interesting exercise to analyse such platitudes and expose the difference between their espoused and actual meanings.  So I sign off for 2013, wishing you  many hours of platitude-deconstructing fun :-)
> getTransformations()
[1] "removeNumbers"     "removePunctuation" "removeWords"       "stemDocument"      "stripWhitespace"  
> #create the toSpace content transformer
> toSpace <- content_transformer(function(x, pattern) {return (gsub(pattern, " ", x))})
> docs <- tm_map(docs, toSpace, "-")
> docs <- tm_map(docs, toSpace, ":")
> #Remove punctuation ? replace punctuation marks with " "
> docs <- tm_map(docs, removePunctuation)
> 
> docs <- tm_map(docs, toSpace, "’")
> docs <- tm_map(docs, toSpace, "‘")
> docs <- tm_map(docs, toSpace, " -")
> #Transform to lower case (need to wrap in content_transformer)
> docs <- tm_map(docs,content_transformer(tolower))
> #Strip digits (std transformation, so no need for content_transformer)
> docs <- tm_map(docs, removeNumbers)
> #remove stopwords using the standard list in tm
> docs <- tm_map(docs, removeWords, stopwords("english"))
> #Strip whitespace (cosmetic?)
> docs <- tm_map(docs, stripWhitespace)
> writeLines(as.character(docs[[30]]))
understanding flexibility ?close view organizational platitude introduction flexibility one buzzwords keeps coming organizational communiques discussions people continually asked display flexibility without ever told term means flexible workplaces flexible attitudes flexible jobs ?word flexible meaning depends context used words used way become platitudes ?empty words make lot noise post analyse platitude flexibility used organisations discussion based paper thomas eriksen entitled mind gap flexibility epistemology rhetoric new work background ?bit organizational platitudes one things struck moved academia industry difference way words phrases used two domains academics one carefully define terms one uses particularly one coining new term whereas business doesnt seem matter words can mean whatever one wants mean ok exaggeration much indeed paul culmsee discuss first chapter heretics guide best practices many terms commonly bandied organizations platitudes understood differently differe... <truncated>
> #load library
> library(SnowballC)
> 
> #Stem document
> docs <- tm_map(docs,stemDocument)
> writeLines(as.character(docs[[30]]))
understand flexibl ?close view organiz platitud introduct flexibl one buzzword keep come organiz communiqu discuss peopl continu ask display flexibl without ever told term mean flexibl workplac flexibl attitud flexibl job ?word flexibl mean depend context use word use way becom platitud ?empti word make lot nois post analys platitud flexibl use organis discuss base paper thoma eriksen entitl mind gap flexibl epistemolog rhetor new work background ?bit organiz platitud one thing struck move academia industri differ way word phrase use two domain academ one care defin term one use particular one coin new term wherea busi doesnt seem matter word can mean whatev one want mean ok exagger much inde paul culmse discuss first chapter heret guid best practic mani term common bandi organ platitud understood differ differ peopl good exampl platitud word govern one manag may see govern larg oversight control wherea anoth might interpret provid guidanc vari interpret can result major differ way two... <truncated>
> docs <- tm_map(docs, content_transformer(gsub), pattern = "organiz", replacement = "organ")
> docs <- tm_map(docs, content_transformer(gsub), pattern = "organis", replacement = "organ")
> docs <- tm_map(docs, content_transformer(gsub), pattern = "andgovern", replacement = "govern")
> docs <- tm_map(docs, content_transformer(gsub), pattern = "inenterpris", replacement = "enterpris")
> docs <- tm_map(docs, content_transformer(gsub), pattern = "team-", replacement = "team")
> dtm <- DocumentTermMatrix(docs)
> dtm
<<DocumentTermMatrix (documents: 30, terms: 3902)>>
Non-/sparse entries: 13979/103081
Sparsity           : 88%
Maximal term length: 48
Weighting          : term frequency (tf)
> inspect(dtm[1:2,1000:1005])
<<DocumentTermMatrix (documents: 2, terms: 6)>>
Non-/sparse entries: 0/12
Sparsity           : 100%
Maximal term length: 7
Weighting          : term frequency (tf)
Sample             :
                                    Terms
Docs                                 critic current cya cynefin david decid
  BeyondEntitiesAndRelationships.txt      0       0   0       0     0     0
  bigdata.txt                             0       0   0       0     0     0
> freq <- colSums(as.matrix(dtm))
> #length should be total number of terms
> length(freq)
[1] 3902
> #create sort order (descending)
> ord <- order(freq,decreasing=TRUE)
> #inspect most frequently occurring terms
> freq[head(ord)]
   one  organ    can  manag   work system 
   325    275    244    230    209    193 
> 
> #inspect least frequently occurring terms
> freq[tail(ord)]   
  therebi timeorgan  uncommit  unionist   willing   workday 
        1         1         1         1         1         1 
> dtmr <-DocumentTermMatrix(docs, control=list(wordLengths=c(4, 20), bounds = list(global = c(3,27))))
> dtmr
<<DocumentTermMatrix (documents: 30, terms: 1294)>>
Non-/sparse entries: 10071/28749
Sparsity           : 74%
Maximal term length: 15
Weighting          : term frequency (tf)
> freqr <- colSums(as.matrix(dtmr))
> #length should be total number of terms
> length(freqr)
[1] 1294
> 
> #create sort order (asc)
> ordr <- order(freqr,decreasing=TRUE)
> 
> #inspect most frequently occurring terms
> freqr[head(ordr)]
  organ   manag    work  system project problem 
    275     230     209     193     185     173 
> 
> #inspect least frequently occurring terms
> freqr[tail(ordr)]
  hmmm struck multin  lower pseudo  gloss 
     3      3      3      3      3      3 
> findFreqTerms(dtmr,lowfreq=80)
 [1] "action"     "approach"   "base"       "busi"       "data"       "design"     "develop"    "differ"     "discuss"    "enterpris"  "exampl"    
[12] "group"      "howev"      "import"     "issu"       "make"       "manag"      "mani"       "model"      "often"      "organ"      "peopl"     
[23] "point"      "practic"    "problem"    "process"    "project"    "question"   "said"       "situat"     "system"     "thing"      "think"     
[34] "time"       "understand" "view"       "well"       "will"       "work"       "chang"      "consult"    "decis"      "even"       "like"      
> findAssocs(dtmr, "project", 0.6)
$project
 inher  manag  handl occurr 
  0.82   0.69   0.68   0.67 

> findAssocs(dtmr, "enterpris", 0.6)
$enterpris
       agil   increment     realist     upfront   technolog        solv     neither    movement       happi       adapt   architect architectur       chanc 
       0.81        0.79        0.77        0.76        0.69        0.68        0.68        0.66        0.66        0.65        0.65        0.65        0.63 
       fine      featur 
       0.63        0.62 

> findAssocs(dtmr, "system", 0.6)
$system
  design   subset    adopt     user   involv  specifi function   intend     step  softwar   specif   intent   compos   depart    phone frequent    today 
    0.78     0.78     0.77     0.75     0.71     0.71     0.70     0.67     0.67     0.67     0.66     0.66     0.66     0.65     0.63     0.62     0.62 
 pattern   author   wherea   cognit 
    0.61     0.60     0.60     0.60 

> wf=data.frame(term=names(freqr),occurrences=freqr)
> library(ggplot2)
> p <- ggplot(subset(wf, freqr>100), aes(term, occurrences))
> p <- p + geom_bar(stat="identity")
> p <- p + theme(axis.text.x=element_text(angle=45, hjust=1))
> p
> #wordcloud
> library(wordcloud)
> #setting the same seed each time ensures consistent look across clouds
> set.seed(42)
> #limit words by specifying min frequency
> wordcloud(names(freqr),freqr, min.freq=70)
> #…add color
> wordcloud(names(freqr),freqr,min.freq=70,colors=brewer.pal(6,"Dark2"))
> dtmr
<<DocumentTermMatrix (documents: 30, terms: 1294)>>
Non-/sparse entries: 10071/28749
Sparsity           : 74%
Maximal term length: 15
Weighting          : term frequency (tf)
> inspect(dtmr[1:5])
Error in x$nrow : $ operator is invalid for atomic vectors
> inspect(dtmr)
<<DocumentTermMatrix (documents: 30, terms: 1294)>>
Non-/sparse entries: 10071/28749
Sparsity           : 74%
Maximal term length: 15
Weighting          : term frequency (tf)
Sample             :
                                       Terms
Docs                                    approach differ exampl manag organ problem project system will work
  BeyondEntitiesAndRelationships.txt          13     14      6     8     8       5       1      6   12    0
  ConditionsOverCauses.txt                    16      2      5     9    14       5       2      4    5   13
  EmergentDesignInEnterpriseIT.txt            13     11      8     6     8      16      17     13   11    4
  FromInformationToKnowledge.txt               2      7     21     7     9      16       4     21    5   10
  MakingSenseOfOrganizationalChange.txt        7      4     10    10    26      14       7      6   12   25
  MakingSenseOfSensemaking.txt                13     17     15     7    12      12      16      9   18   12
  RoutinesAndReality.txt                       1      7      2     3    10       6       4     36    8   13
  SixHeresiesForBI.txt                         2      2      7     3     7       4       0      4    6    5
  TheEssenceOfEntrepreneurship.txt             7     15     15     2     2       5       1      0   10   10
  ThreeTypesOfUncertainty.txt                 20      6     15     9     3      15       1      0    6    7
> dtm
<<DocumentTermMatrix (documents: 30, terms: 3902)>>
Non-/sparse entries: 13979/103081
Sparsity           : 88%
Maximal term length: 48
Weighting          : term frequency (tf)
> dtmr
<<DocumentTermMatrix (documents: 30, terms: 1294)>>
Non-/sparse entries: 10071/28749
Sparsity           : 74%
Maximal term length: 15
Weighting          : term frequency (tf)
> inspect(dtm)
<<DocumentTermMatrix (documents: 30, terms: 3902)>>
Non-/sparse entries: 13979/103081
Sparsity           : 88%
Maximal term length: 48
Weighting          : term frequency (tf)
Sample             :
                                       Terms
Docs                                    can manag one organ problem project system use way work
  BeyondEntitiesAndRelationships.txt     26     8  15     8       5       1      6  18   9    0
  ConditionsOverCauses.txt                8     9   7    14       5       2      4   1   3   13
  EmergentDesignInEnterpriseIT.txt       15     6  28     8      16      17     13  14  11    4
  FromInformationToKnowledge.txt         35     7  17     9      16       4     21  25   8   10
  MakingSenseOfOrganizationalChange.txt  15    10  26    26      14       7      6  10  27   25
  MakingSenseOfSensemaking.txt           25     7  26    12      12      16      9  27  21   12
  RoutinesAndReality.txt                  8     3  13    10       6       4     36  16   9   13
  SixHeresiesForBI.txt                    6     3  10     7       4       0      4   9   2    5
  TheEssenceOfEntrepreneurship.txt        5     2  24     2       5       1      0   6  30   10
  ThreeTypesOfUncertainty.txt            13     9  18     3      15       1      0   3   4    7
> Sys.setlocale(category = "LC_ALL", locale = "US")
[1] "LC_COLLATE=English_United States.1252;LC_CTYPE=English_United States.1252;LC_MONETARY=English_United States.1252;LC_NUMERIC=C;LC_TIME=English_United States.1252"
> library(tm)
> #Create Corpus
> docs <- Corpus(DirSource("D:/Users/Hyo/Documents/TextMining"))
> docs
<<SimpleCorpus>>
Metadata:  corpus specific: 1, document level (indexed): 0
Content:  documents: 30
> #inspect a particular document
> writeLines(as.character(docs[[30]]))
Understanding "flexibility" ?a close-up view of an organizational platitude

Introduction
Flexibility is one of those buzzwords that keeps coming up in organizational communiques and discussions. People are continually asked to display flexibility, without ever being told what the term means:  flexible workplaces, flexible attitudes, flexible jobs ?the word itself has a flexible meaning that depends on the context in which it is used and by whom.
When words are used in this way they become platitudes ?empty words that make a lot of noise. In this post, I analyse the platitude, flexibility, as it is used in organisations. My discussion is based on a paper by Thomas Eriksen entitled, Mind the Gap: Flexibility, Epistemology and the Rhetoric of New Work.
Background ?a bit about organizational platitudes
One of the things that struck me when I moved from academia to industry is the difference in the way words or phrases are used in the two domains. In academics one has to carefully define the terms one uses (particularly if one is coining a new term) whereas in business it doesn't seem to matter, words can mean whatever one wants them to mean (OK, this is an exaggeration, but not by too much). Indeed, as Paul Culmsee and I discuss in the first chapter of The Heretic's Guide to Best Practices, many terms that are commonly bandied about in organizations are platitudes because they are understood differently by different people.
A good example of a platitude is the word governance. One manager may see governance as being largely about oversight and control whereas another might interpret it as being about providing guidance.  Such varying interpretations can result in major differences in the way the two managers implement governance:  the first one might enforce it as a compliance-oriented set of processes that leave little room for individual judgement while the other might implement it as a broad set of guidelines that leave many of the decisions in the hands of those who are actually doing the work. Needless to say, the results in the two cases are likely to be different too.
Flexibility ?the conventional view
A good place to start our discussion of flexibility is with the dictionary. The online Oxford Dictionary defines at as:
Flexibility (noun):
 the ability to be easily modified
 willingness to change or compromise
The term is widely used in both these senses in organizational settings. For example, people speak of flexible designs (i.e. designs that can be easily modified) or flexible people (referring to those who are willing to change or compromise). However,  and this is the problem:  the term is open to interpretation ?what Jack might term a flexible approach may be seen by Jill as a complete lack of method. These differences in interpretation become particularly obvious when the word is used in a broad context ?such as in a statement justifying an organizational change.  An executive might see a corporate restructure and the resulting changes in jobs/roles as a means to achieve organizational flexibility, but those affected by it may see it as constraining theirs.  As Eriksen states:
Jobs are flexible in the sense that they are unstable and uncertain, few employees hold the same jobs for many years, the content of jobs can be changed almost overnight, and the boundaries between work and leisure are negotiable and chronically fuzzy.
Indeed, such "flexibility" which requires one to change at short notice results in a fragmentation of individual experience and a resulting loss of a coherent narrative of one's life. It appears that increased flexibility in one aspect results in a loss of flexibility in another. Any sensible definition of flexibility ought to reflect this.
Understanding flexibility
Consider the following definition of flexibility proposed by Gregory Bateson:
"Flexibility is uncommitted potential for change"
This deceptively simple statement is a good place to start understanding what flexibility really means for projects, organisations 꿢nd even software systems.
As Eriksen tells us, Bateson proposed this definition in the context of ecology. In particular, Bateson had in mind the now obvious notion that the increased flexibility we gain through our increasingly energy-hungry lifestyles results in a decrease in the environment's capacity to cope with the consequences. This is true of flexibility in any context: a gain in flexibility in one dimension will necessarily be accompanied by a loss of flexibility in another.
Another implication of the above definition is that a system that is running at or near the limits of its operating variables cannot be flexible.  The following examples should make this clear:
 A project team that is putting in 18 hour workdays in order to finish a project on time.
 A car that's being driven at top speed.
 A family living beyond their means.
All these systems are operating at or near their limits, they have little or no spare capacity to accommodate change.
A third implication of the definition follows from the preceding one:  the key variables of a flexible system should lie in the mid-range of their upper and lower limits. In terms of above examples:
 The project team should be putting in normal hours.
 The car should be driven at or below the posted road speed limits
 The family should be living within its income, with a reasonable amount to spare.
Of course, the whole point of ensuring that systems operate in their comfort zone is that they can be revved up if the need arises. Such revving up, however,  should be an exceptional circumstance rather than the norm ?a point that those who run projects, organisations (and, yes, even vehicles) often tend to forget. If one operates a system at the limits of its tolerance for too long, not only will it not be flexible, it will break.
Flexibility in the workplace
As mentioned in the introduction, the term flexibility keeps cropping up in organizational settings: corporate communiques exhort employees to be flexible in the face of change.  This is typically a coded signal that employees should expect uncertainty and be prepared to adjust to it.  A related manifestation of flexibility is the blurring of the distinction between work and personal life. As Eriksen puts it:
The term flexibility is often used to describe this new situation: Jobs are flexible in the sense that they are unstable and uncertain, few employees hold the same jobs for many years, the content of jobs can be changed, and the boundaries between work and leisure are poorly defined.
This trend is aided by recent developments in technology that enable employees to be perpetually on call. This is often sold as a work from home initiative but usually ends up being much more.  Eriksen has this to say about home offices:
One recent innovation typically associated with flexibility is the home office. In Scandinavia (and some other prosperous, technologically optimistic regions), many companies equipped some of their employees with home computers with online access to the company network in the early 1990s, in order to enhance their flexibility. This was intended to enable employees to work from home part of the time, thereby making the era when office workers were chained to the office desk all day obsolete.
In the early days, there were widespread worries among employers to the effect that a main outcome of this new flexibility would consist in a reduction of productivity. Since there was no legitimate way of checking how the staff actually spent their time out of the office, it was often suspected that they worked less from home than they were supposed to. If this were in fact the case, working from home would have led to a real increase in the flexibility of time budgeting. However, work researchers eventually came up with a different picture. By the late 1990s, hardly anybody spoke of the home office as a convenient way of escaping from work; rather, the concern among unionists as well as researchers was now that increasing numbers of employees were at pains to distinguish between working hours and leisure time, and were suffering symptoms of burnout and depression. The home office made it difficult to distinguish between contexts that were formerly mutually exclusive because of differ... <truncated>
It is interesting to see this development in the light of Bateson's definition of flexibility: the employee gains flexibility in space (he or she can work from home or from the office) at the expense of flexibility in time(organization time encroaches on personal time). As Eriksen states:
 There seems to be a classic Batesonian flexibility trade-off associated with the new information technologies: increased spatial flexibility entails decreased temporal flexibility. If inaccessibility and 'empty time' are understood as scarce resources, the context of 'new work' thus seems to be an appropriate context for a new economics as well. In fact, a main environmental challenge of our near future will consist in protecting slow time and gaps from environmental degradation.
In short, it appears that flexibility for the organization necessarily implies a loss of flexibility for the individual.
Conclusion
Flexibility is in the eye of the beholder: an action to increase organisational flexibility by, say, redeploying employees would likely be seen by those affected as a move that constrains their  (individual) flexibility.  Such a dual meaning is characteristic of many organizational platitudes such as Excellence, Synergy andGovernance. It is an interesting exercise to analyse such platitudes and expose the difference between their espoused and actual meanings.  So I sign off for 2013, wishing you  many hours of platitude-deconstructing fun :-)
> getTransformations()
[1] "removeNumbers"     "removePunctuation" "removeWords"       "stemDocument"     
[5] "stripWhitespace"  
> #create the toSpace content transformer
> toSpace <- content_transformer(function(x, pattern) {return (gsub(pattern, " ", x))})
> docs <- tm_map(docs, toSpace, "-")
> docs <- tm_map(docs, toSpace, ":")
> #Remove punctuation ? replace punctuation marks with " "
> docs <- tm_map(docs, removePunctuation)
> 
> docs <- tm_map(docs, toSpace, "’")
> docs <- tm_map(docs, toSpace, "‘")
> docs <- tm_map(docs, toSpace, " -")
> #Transform to lower case (need to wrap in content_transformer)
> docs <- tm_map(docs,content_transformer(tolower))
> #Strip digits (std transformation, so no need for content_transformer)
> docs <- tm_map(docs, removeNumbers)
> #remove stopwords using the standard list in tm
> docs <- tm_map(docs, removeWords, stopwords("english"))
> #Strip whitespace (cosmetic?)
> docs <- tm_map(docs, stripWhitespace)
> writeLines(as.character(docs[[30]]))
understanding flexibility ?close view organizational platitude introduction flexibility one buzzwords keeps coming organizational communiques discussions people continually asked display flexibility without ever told term means flexible workplaces flexible attitudes flexible jobs ?word flexible meaning depends context used words used way become platitudes ?empty words make lot noise post analyse platitude flexibility used organisations discussion based paper thomas eriksen entitled mind gap flexibility epistemology rhetoric new work background ?bit organizational platitudes one things struck moved academia industry difference way words phrases used two domains academics one carefully define terms one uses particularly one coining new term whereas business doesnt seem matter words can mean whatever one wants mean ok exaggeration much indeed paul culmsee discuss first chapter heretics guide best practices many terms commonly bandied organizations platitudes understood differently differe... <truncated>
> #load library
> library(SnowballC)
> 
> #Stem document
> docs <- tm_map(docs,stemDocument)
> writeLines(as.character(docs[[30]]))
understand flexibl ?close view organiz platitud introduct flexibl one buzzword keep come organiz communiqu discuss peopl continu ask display flexibl without ever told term mean flexibl workplac flexibl attitud flexibl job ?word flexibl mean depend context use word use way becom platitud ?empti word make lot nois post analys platitud flexibl use organis discuss base paper thoma eriksen entitl mind gap flexibl epistemolog rhetor new work background ?bit organiz platitud one thing struck move academia industri differ way word phrase use two domain academ one care defin term one use particular one coin new term wherea busi doesnt seem matter word can mean whatev one want mean ok exagger much inde paul culmse discuss first chapter heret guid best practic mani term common bandi organ platitud understood differ differ peopl good exampl platitud word govern one manag may see govern larg oversight control wherea anoth might interpret provid guidanc vari interpret can result major differ way two... <truncated>
> docs <- tm_map(docs, content_transformer(gsub), pattern = "organiz", replacement = "organ")
> docs <- tm_map(docs, content_transformer(gsub), pattern = "organis", replacement = "organ")
> docs <- tm_map(docs, content_transformer(gsub), pattern = "andgovern", replacement = "govern")
> docs <- tm_map(docs, content_transformer(gsub), pattern = "inenterpris", replacement = "enterpris")
> docs <- tm_map(docs, content_transformer(gsub), pattern = "team-", replacement = "team")
> dtm <- DocumentTermMatrix(docs)
> dtm
<<DocumentTermMatrix (documents: 30, terms: 3902)>>
Non-/sparse entries: 13979/103081
Sparsity           : 88%
Maximal term length: 48
Weighting          : term frequency (tf)
> inspect(dtm[1:2,1000:1005])
<<DocumentTermMatrix (documents: 2, terms: 6)>>
Non-/sparse entries: 0/12
Sparsity           : 100%
Maximal term length: 7
Weighting          : term frequency (tf)
Sample             :
                                    Terms
Docs                                 critic current cya cynefin david decid
  BeyondEntitiesAndRelationships.txt      0       0   0       0     0     0
  bigdata.txt                             0       0   0       0     0     0
> freq <- colSums(as.matrix(dtm))
> #length should be total number of terms
> length(freq)
[1] 3902
> #create sort order (descending)
> ord <- order(freq,decreasing=TRUE)
> #inspect most frequently occurring terms
> freq[head(ord)]
   one  organ    can  manag   work system 
   325    275    244    230    209    193 
> 
> #inspect least frequently occurring terms
> freq[tail(ord)]   
  therebi timeorgan  uncommit  unionist   willing   workday 
        1         1         1         1         1         1 
> dtmr <-DocumentTermMatrix(docs, control=list(wordLengths=c(4, 20), bounds = list(global = c(3,27))))
> dtmr
<<DocumentTermMatrix (documents: 30, terms: 1294)>>
Non-/sparse entries: 10071/28749
Sparsity           : 74%
Maximal term length: 15
Weighting          : term frequency (tf)
> freqr <- colSums(as.matrix(dtmr))
> #length should be total number of terms
> length(freqr)
[1] 1294
> 
> #create sort order (asc)
> ordr <- order(freqr,decreasing=TRUE)
> 
> #inspect most frequently occurring terms
> freqr[head(ordr)]
  organ   manag    work  system project problem 
    275     230     209     193     185     173 
> 
> #inspect least frequently occurring terms
> freqr[tail(ordr)]
  hmmm struck multin  lower pseudo  gloss 
     3      3      3      3      3      3 
> findFreqTerms(dtmr,lowfreq=80)
 [1] "action"     "approach"   "base"       "busi"       "data"       "design"    
 [7] "develop"    "differ"     "discuss"    "enterpris"  "exampl"     "group"     
[13] "howev"      "import"     "issu"       "make"       "manag"      "mani"      
[19] "model"      "often"      "organ"      "peopl"      "point"      "practic"   
[25] "problem"    "process"    "project"    "question"   "said"       "situat"    
[31] "system"     "thing"      "think"      "time"       "understand" "view"      
[37] "well"       "will"       "work"       "chang"      "consult"    "decis"     
[43] "even"       "like"      
> findAssocs(dtmr, "project", 0.6)
$project
 inher  manag  handl occurr 
  0.82   0.69   0.68   0.67 

> findAssocs(dtmr, "enterpris", 0.6)
$enterpris
       agil   increment     realist     upfront   technolog        solv     neither 
       0.81        0.79        0.77        0.76        0.69        0.68        0.68 
   movement       happi       adapt   architect architectur       chanc        fine 
       0.66        0.66        0.65        0.65        0.65        0.63        0.63 
     featur 
       0.62 

> findAssocs(dtmr, "system", 0.6)
$system
  design   subset    adopt     user   involv  specifi function   intend     step  softwar 
    0.78     0.78     0.77     0.75     0.71     0.71     0.70     0.67     0.67     0.67 
  specif   intent   compos   depart    phone frequent    today  pattern   author   wherea 
    0.66     0.66     0.66     0.65     0.63     0.62     0.62     0.61     0.60     0.60 
  cognit 
    0.60 

> wf=data.frame(term=names(freqr),occurrences=freqr)
> library(ggplot2)
> p <- ggplot(subset(wf, freqr>100), aes(term, occurrences))
> p <- p + geom_bar(stat="identity")
> p <- p + theme(axis.text.x=element_text(angle=45, hjust=1))
> p
> #wordcloud
> library(wordcloud)
> #setting the same seed each time ensures consistent look across clouds
> set.seed(42)
> #limit words by specifying min frequency
> wordcloud(names(freqr),freqr, min.freq=70)
> #…add color
> wordcloud(names(freqr),freqr,min.freq=70,colors=brewer.pal(6,"Dark2"))
</code>



====== E.g. 2 mode matrix data ======
{{:termDocMatrix.rdata}}

__Load data__
<code r>setwd("d:/rdata")
load("data/termDocMatrix.rdata") # load termDocMatrix
termDocMatrix[5:10,1:20] # inspect part of the matrix
</code>
<code>> load("termDocMatrix.rdata") # load termDocMatrix
> termDocMatrix[5:10,1:20] # inspect part of the matrix
              Docs
Terms          1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
  data         1 1 0 0 1 0 0 0 0  0  1  1  1  1  1  0  1  0  0  0
  examples     0 0 0 0 0 0 0 0 0  0  0  0  0  0  0  0  0  0  0  0
  introduction 0 0 0 0 0 0 0 0 0  0  0  0  0  0  0  0  0  0  0  1
  mining       0 0 0 0 0 0 0 0 0  0  0  1  1  0  1  0  0  0  0  0
  network      0 0 0 0 0 0 0 0 0  0  0  0  0  0  0  1  0  1  1  1
  package      0 0 0 1 1 0 0 0 0  0  0  1  0  0  0  0  0  0  0  0
</code>

Terms x Documents matrix data = two mode matrix data

__Transform Data into an Adjacency Matrix__
<code>termDocMatrix[termDocMatrix>=1] <- 1 # change it to a Boolean matrix
termMatrix <- termDocMatrix %*% t(termDocMatrix) # transform into a term-term adjacency matrix
termMatrix[5:10,5:10] # inspect terms numbered 5 to 10</code> 

<code>> termDocMatrix[termDocMatrix>=1] <- 1 # change it to a Boolean matrix
> termMatrix <- termDocMatrix %*% t(termDocMatrix) # transform into a term-term adjacency matrix
> termMatrix[5:10,5:10] # inspect terms numbered 5 to 10 
              Terms
Terms          data examples introduction mining network package
  data           53        5            2     34       0       7
  examples        5       17            2      5       2       2
  introduction    2        2           10      2       2       0
  mining         34        5            2     47       1       5
  network         0        2            2      1      17       1
  package         7        2            0      5       1      21
> </code>

Two mode -> one mode data by ''termMatrix = termDocmatrix * transposed(termDocmatrix)''
  * termMatrix data = one mode matrix data showing the relationships among the words (appeared in the Doc)
  * For example, the word, "data" appears in the doc a toal of "53" 
  * And, In a total of "5" cases, both **data** and **examples** appeared at the same time(document).
  * Therefore, the word, "**mining(34)**" is more close (relevant) to "data" than "**examples(5)**"

Next we can build a graph with graph.adjacency() from package igraph.

<code>library(igraph)
# build a graph from the above matrix
g <- graph.adjacency(termMatrix, weighted=T, mode = “undirected”) 
# remove loops
g <- simplify(g)
# set labels and degrees of vertices
V(g)$label <- V(g)$name 
V(g)$degree <- degree(g)
V(g)$label
V(g)$degree
</code>

<code>> library(igraph)
> g <- graph.adjacency(termMatrix, weighted=T, mode = "undirected") 
> g <- simplify(g)
> V(g)$label <- V(g)$name 
> V(g)$degree <- degree(g)
> V(g)$degree
 [1] 17  6  9  9 18 14 12 20 14 13  8  7  8 17  9 11 15 11 11 16 15
> V(g)$label
 [1] "analysis"     "applications" "code"         "computing"   
 [5] "data"         "examples"     "introduction" "mining"      
 [9] "network"      "package"      "parallel"     "positions"   
[13] "postdoctoral" "r"            "research"     "series"      
[17] "slides"       "social"       "time"         "tutorial"    
[21] "users"        
</code>

__Plot a Graph__
<code># set seed to make the layout reproducible
set.seed(3952)
layout1 <- layout.fruchterman.reingold(g)
plot(g, layout=layout1)
</code>

<code>> # set seed to make the layout reproducible
> set.seed(3952)
> layout1 <- layout.fruchterman.reingold(g)
> plot(g, layout=layout1)
</code>
{{wordsBywords.png}}

__Different layout of plot__
<code>plot(g, layout=layout.kamada.kawai)
tkplot(g, layout=layout.kamada.kawai)</code>

{{terms.jpg}}
<code>CONCOR
--------------------------------------------------------------------------------

Diagonal:                               Ignore
Max partitions:                         3
Input dataset:                          terms (D:\Users\Hyo\Documents\UCINET data\rdm\terms)

Initial Correlation Matrix

                      1     2     3     4     5     6     7     8     9    10    11    12    13    14    15    16    17    18    19    20    21
                  analy appli  code compu  data examp intro minin netwo packa paral posit postd     r resea serie slide socia  time tutor users
                  ----- ----- ----- ----- ----- ----- ----- ----- ----- ----- ----- ----- ----- ----- ----- ----- ----- ----- ----- ----- -----
  1     analysis   1.00  0.19  0.49  0.22  0.18  0.51  0.73  0.23  0.75  0.43  0.13  0.13  0.10  0.07  0.02  0.26  0.59  0.71  0.26  0.73  0.47
  2 applications   0.19  1.00  0.36  0.28  0.97  0.56  0.38  0.98 -0.14  0.68  0.26  0.38  0.50  0.72  0.50  0.17  0.63 -0.16  0.17  0.55  0.51
  3         code   0.49  0.36  1.00  0.45  0.53  0.92  0.47  0.38  0.13  0.67  0.35 -0.22 -0.17  0.40 -0.22  0.55  0.67  0.05  0.55  0.62  0.75
  4    computing   0.22  0.28  0.45  1.00  0.35  0.56  0.03  0.32  0.03  0.76  0.97 -0.22 -0.27  0.20 -0.19  0.16  0.61 -0.11  0.16  0.44  0.71
  5         data   0.18  0.97  0.53  0.35  1.00  0.56  0.28  0.94 -0.00  0.64  0.30 -0.00  0.29  0.54  0.18  0.28  0.59 -0.06  0.28  0.49  0.60
  6     examples   0.51  0.56  0.92  0.56  0.56  1.00  0.47  0.60  0.26  0.83  0.47 -0.11 -0.07  0.68 -0.10  0.54  0.85  0.11  0.54  0.76  0.84
  7 introduction   0.73  0.38  0.47  0.03  0.28  0.47  1.00  0.37  0.48  0.42 -0.01  0.07  0.17  0.52 -0.01  0.41  0.67  0.36  0.41  0.68  0.48
  8       mining   0.23  0.98  0.38  0.32  0.94  0.60  0.37  1.00 -0.05  0.71  0.28  0.41  0.31  0.67  0.51  0.19  0.62 -0.13  0.19  0.52  0.54
  9      network   0.75 -0.14  0.13  0.03 -0.00  0.26  0.48 -0.05  1.00  0.14 -0.03  0.13  0.21 -0.09 -0.01  0.15  0.24  0.92  0.15  0.39  0.25
 10      package   0.43  0.68  0.67  0.76  0.64  0.83  0.42  0.71  0.14  1.00  0.68 -0.01  0.02  0.84  0.03  0.32  0.88 -0.02  0.32  0.77  0.91
 11     parallel   0.13  0.26  0.35  0.97  0.30  0.47 -0.01  0.28 -0.03  0.68  1.00 -0.28 -0.21  0.32 -0.23  0.10  0.54 -0.16  0.10  0.43  0.65
 12    positions   0.13  0.38 -0.22 -0.22 -0.00 -0.11  0.07  0.41  0.13 -0.01 -0.28  1.00  0.90 -0.00  0.94 -0.26 -0.06  0.30 -0.26  0.01 -0.21
 13 postdoctoral   0.10  0.50 -0.17 -0.27  0.29 -0.07  0.17  0.31  0.21  0.02 -0.21  0.90  1.00  0.15  0.87 -0.19  0.01  0.28 -0.19  0.09 -0.15
 14            r   0.07  0.72  0.40  0.20  0.54  0.68  0.52  0.67 -0.09  0.84  0.32 -0.00  0.15  1.00  0.13  0.22  0.72 -0.16  0.22  0.74  0.80
 15     research   0.02  0.50 -0.22 -0.19  0.18 -0.10 -0.01  0.51 -0.01  0.03 -0.23  0.94  0.87  0.13  1.00 -0.30 -0.06  0.05 -0.30 -0.01 -0.18
 16       series   0.26  0.17  0.55  0.16  0.28  0.54  0.41  0.19  0.15  0.32  0.10 -0.26 -0.19  0.22 -0.30  1.00  0.55  0.01  1.00  0.33  0.49
 17       slides   0.59  0.63  0.67  0.61  0.59  0.85  0.67  0.62  0.24  0.88  0.54 -0.06  0.01  0.72 -0.06  0.55  1.00  0.14  0.55  0.81  0.92
 18       social   0.71 -0.16  0.05 -0.11 -0.06  0.11  0.36 -0.13  0.92 -0.02 -0.16  0.30  0.28 -0.16  0.05  0.01  0.14  1.00  0.01  0.41  0.12
 19         time   0.26  0.17  0.55  0.16  0.28  0.54  0.41  0.19  0.15  0.32  0.10 -0.26 -0.19  0.22 -0.30  1.00  0.55  0.01  1.00  0.33  0.49
 20     tutorial   0.73  0.55  0.62  0.44  0.49  0.76  0.68  0.52  0.39  0.77  0.43  0.01  0.09  0.74 -0.01  0.33  0.81  0.41  0.33  1.00  0.80
 21        users   0.47  0.51  0.75  0.71  0.60  0.84  0.48  0.54  0.25  0.91  0.65 -0.21 -0.15  0.80 -0.18  0.49  0.92  0.12  0.49  0.80  1.00



PARTITION DIAGRAM

          i         a                           p
          n         p                           o
          t         p             c           p s
        a r         l       p   t o   e     r o t
        n o         i     p a   u m   x   n e s d
        a d     s   c m   a r s t p   a s e s i o
        l u     e   a i   c a l o u u m o t e t c
        y c c t r d t n   k l i r t s p c w a i t
        s t o i i a i i   a l d i i e l i o r o o
        i i d m e t o n   g e e a n r e a r c n r
        s o e e s a n g r e l s l g s s l k h s a

              1 1       1 1 1 1 2   2   1   1 1 1
Level   1 7 3 9 6 5 2 8 4 0 1 7 0 4 1 6 8 9 5 2 3
-----   - - - - - - - - - - - - - - - - - - - - -
    3   XXX XXXXX XXXXXXX XXXXXXXXXXXXX XXX XXX .
    2   XXXXXXXXX XXXXXXXXXXXXXXXXXXXXX XXX XXXXX
    1   XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX XXXXXXXXX



Relation Sheet 1
Blocked Matrix

                    1  7    3 19 16    5  2  8 14   10 11 17 20  4 21  6   18  9   15 12   13  
                   an in   co ti se   da ap mi  r   pa pa sl tu co us ex   so ne   re po   po  
                  ---------------------------------------------------------------------------- 
  1     analysis | 23  2 |  1  4  4 |  4     4 11 |  2     3  4     5  4 |  9 12 |  1  2 |  3 |
  7 introduction |  2 10 |     1  1 |  2     2  2 |        2  1     2  2 |  2  2 |       |    |
                 ------------------------------------------------------------------------------
  3         code |  1    |  9  2  2 |  1     3  8 |           1        6 |     1 |       |    |
 19         time |  4  1 |  2  8  8 |  1     3  5 |        2  1     2  2 |       |       |    |
 16       series |  4  1 |  2  8  8 |  1     3  5 |        2  1     2  2 |       |       |    |
                 ------------------------------------------------------------------------------
  5         data |  4  2 |  1  1  1 | 53  7 34 22 |  7  1  4  4  1  4  5 |       |  6  5 |  5 |
  2 applications |       |          |  7  9  6  4 |  1              1    |       |  1    |    |
  8       mining |  4  2 |  3  3  3 | 34  6 47 20 |  5  1  4  4  1  5  5 |  1  1 |  2  1 |  4 |
 14            r | 11  2 |  8  5  5 | 22  4 20 70 | 15  7  9  7  9 15 14 |  3  6 |       |    |
                 ------------------------------------------------------------------------------
 10      package |  2    |          |  7  1  5 15 | 21  3  1  4  2  5  2 |     1 |  1    |    |
 11     parallel |       |          |  1     1  7 |  3  8  1  1  7  2    |       |       |    |
 17       slides |  3  2 |     2  2 |  4     4  9 |  1  1 16  1  1  4  1 |  1  2 |       |    |
 20     tutorial |  4  1 |  1  1  1 |  4     4  7 |  4  1  1 16  1  3  3 |  2  5 |       |    |
  4    computing |       |          |  1     1  9 |  2  7  1  1 10  2    |       |       |  1 |
 21        users |  5  2 |     2  2 |  4  1  5 15 |  5  2  4  3  2 18  3 |     2 |       |    |
  6     examples |  4  2 |  6  2  2 |  5     5 14 |  2     1  3     3 17 |  1  2 |       |    |
                 ------------------------------------------------------------------------------
 18       social |  9  2 |          |        1  3 |        1  2        1 | 12 11 |  2  2 |  3 |
  9      network | 12  2 |  1       |        1  6 |  1     2  5     2  2 | 11 17 |  1  2 |  2 |
                 ------------------------------------------------------------------------------
 15     research |  1    |          |  6  1  2    |  1                   |  2  1 | 12  4 |  4 |
 12    positions |  2    |          |  5     1    |                      |  2  2 |  4 11 |  4 |
                 ------------------------------------------------------------------------------
 13 postdoctoral |  3    |          |  5     4    |              1       |  3  2 |  4  4 | 11 |
                  -----------------------------------------------------------------------------



Density Matrix

             1      2      3      4      5      6      7
        ------ ------ ------ ------ ------ ------ ------
    1    2.000  1.833  3.125  1.786  6.250  0.750  1.500
    2    1.833  4.000  2.500  1.000  0.167  0.000  0.000
    3    3.125  2.500 15.500  4.607  1.375  1.875  2.250
    4    1.786  1.000  4.607  2.238  1.143  0.071  0.143
    5    6.250  0.167  1.375  1.143 11.000  1.750  2.500
    6    0.750  0.000  1.875  0.071  1.750  4.000  4.000
    7    1.500  0.000  2.250  0.143  2.500  4.000       

R-squared = 0.474

First order actor-by-actor correlation matrix saved as dataset Concor1stCorr
Partition-by-actor indicator matrix saved as dataset ConcorCCPart
Permutation vector saved as dataset ConcorCCPerm

----------------------------------------
Running time:  00:00:01
Output generated:  08 12 16 09:32:55
UCINET 6.614 Copyright (c) 1992-2016 Analytic Technologies

</code>

{{:terms_dendo.jpg}}

====== E.g. Dan McFarland's students data ======
data file 
  * {{:mag_act96.txt}}
  * {{:mag_act97.txt}}
  * {{:mag_act98.txt}}

<code>

</code>

<code># Load the 'igraph' library
library('igraph')

# (1) Read in the data files, NA data objects coded as 'na'
magact96 = read.delim('http://commres.net/wiki/_media/mag_act96.txt', na.strings = 'na')
magact97 = read.delim('http://commres.net/wiki/_media/mag_act97.txt', na.strings = 'na')
magact98 = read.delim('http://commres.net/wiki/_media/mag_act98.txt', na.strings = 'na')</code>

Variables: 
  * ID, gender(GND), grade(GRD), race(RCE)
  * Clubs attended by the ID (1 if so, 0 if not so): Asian.Club, Hispanic.Club, . . . . 


<code>magattrib = magact96[,1:4]

g96 <- as.matrix(magact96[,-(1:4)]); row.names(g96) = magact96$ID.
g97 <- as.matrix(magact97[,-(1:4)]); row.names(g97) = magact97$ID.
g98 <- as.matrix(magact98[,-(1:4)]); row.names(g98) = magact98$ID.</code>

<code>i96 <- graph.incidence(g96, mode=c('all') )
i97 <- graph.incidence(g97, mode=c('all') )
i98 <- graph.incidence(g98, mode=c('all') )</code>

<code>V(i96)$color[1:1295] <- rgb(1,0,0,.5)
V(i96)$color[1296:1386] <- rgb(0,1,0,.5)</code>

<code>V(i96)$label <- V(i96)$name
V(i96)$label.color <- rgb(0,0,.2,.5)
V(i96)$label.cex <- .4
V(i96)$size <- 6
V(i96)$frame.color <- NA</code>

<code>E(i96)$color <- rgb(.5,.5,0,.2)</code>

<code>pdf('i96.pdf')
plot(i96, layout=layout.fruchterman.reingold)
dev.off()</code>

{{:i96.pdf}}

<code>i96 <- delete.vertices(i96, V(i96)[ degree(i96)==0 ])
V(i96)$label[1:857] <- NA
V(i96)$color[1:857] <-  rgb(1,0,0,.1)
V(i96)$size[1:857] <- 2
 
E(i96)$width <- .3
E(i96)$color <- rgb(.5,.5,0,.1)
 
pdf('i96.2.pdf')
plot(i96, layout=layout.kamada.kawai)
dev.off()
 
pdf('i96.3.pdf')
plot(i96, layout=layout.fruchterman.reingold.grid)
dev.off()
 
pdf('i96.4.pdf')
plot(i96, layout=layout.fruchterman.reingold)
dev.off()</code>

{{:i96.2.pdf}}
{{:i96.3.pdf}}
{{:i96.4.pdf}}

<code>g96e <- t(g96) %*% g96
g97e <- t(g97) %*% g97
g98e <- t(g98) %*% g98
 
i96e <- graph.adjacency(g96e, mode = 'undirected')
</code>

<code>E(i96e)$weight <- count.multiple(i96e)
i96e <- simplify(i96e)</code>

<code># Set vertex attributes
V(i96e)$label <- V(i96e)$name
V(i96e)$label.color <- rgb(0,0,.2,.8)
V(i96e)$label.cex <- .6
V(i96e)$size <- 6
V(i96e)$frame.color <- NA
V(i96e)$color <- rgb(0,0,1,.5)
 
# Set edge gamma according to edge weight
egam <- (log(E(i96e)$weight)+.3)/max(log(E(i96e)$weight)+.3)
E(i96e)$color <- rgb(.5,.5,0,egam)</code>

<code>pdf('i96e.pdf')
plot(i96e, main = 'layout.kamada.kawai', layout=layout.kamada.kawai)
plot(i96e, main = 'layout.fruchterman.reingold', layout=layout.fruchterman.reingold)
dev.off()</code>

{{:i96e.pdf}}

===== Group overlap networks and plots =====

<code>ol96 <- g96e/diag(g96e)
ol97 <- g97e/diag(g97e)
ol98 <- g98e/diag(g98e)</code>

<code>magall <- ol96 + ol97 + ol98
magall[is.na(magall)] <- 0</code>

<code>magdiag <- apply(cbind(diag(g96e), diag(g97e), diag(g98e)), 1, mean )</code>

<code>magallg <- graph.adjacency(magall, weighted=T)
 
# Degree
V(magallg)$degree <- degree(magallg)
 
# Betweenness centrality
V(magallg)$btwcnt <- betweenness(magallg)</code>

<code>plot(density(magall))</code>

<code>magallgt1 <- magall
magallgt1[magallgt1 < 1] <- 0
magallggt1 <- graph.adjacency(magallgt1, weighted=T)
 
# Removes loops:
magallggt1 <- simplify(magallggt1, remove.multiple=FALSE, remove.loops=TRUE)
</code>

<code>magallggt1$layout <- layout.fruchterman.reingold(magallggt1)
V(magallggt1)$label <- V(magallggt1)$name
tkplot(magallggt1)</code>

<code>magallggt1$layout <- tkplot.getcoords(1)</code>

<code># Set vertex attributes
V(magallggt1)$label <- V(magallggt1)$name
V(magallggt1)$label.color <- rgb(0,0,.2,.6)
V(magallggt1)$size <- 6
V(magallggt1)$frame.color <- NA
V(magallggt1)$color <- rgb(0,0,1,.5)
 
# Set edge attributes
E(magallggt1)$arrow.size <- .3
 
# Set edge gamma according to edge weight
egam <- (E(magallggt1)$weight+.1)/max(E(magallggt1)$weight+.1)
E(magallggt1)$color <- rgb(.5,.5,0,egam)</code>

<code>V(magallggt1)$label.cex <- V(magallggt1)$degree/(max(V(magallggt1)$degree)/2)+ .3
#note, unfortunately one must play with the formula above to get the
#ratio just right</code>

<code>pdf('magallggt1customlayout.pdf')
plot(magallggt1)
dev.off()</code>