Differences

This shows you the differences between two versions of the page.

--- c:itamc:2018 [2018/11/06 17:45] – created hkimscil
+++ c:itamc:2018 [2019/11/12 15:43] (current) – hkimscil
@@ Line 8: / Line 8: @@
 create Textmining directory in R working directory.
 Unzip the zip file.
-====== e.g. 1 ======
+====== Text mining e.g. 1 ======
 <code>
 NeededPackages <- c("tm", "SnowballC", "RColorBrewer", "ggplot2", "wordcloud", "biclust",
@@ Line 97: / Line 97: @@
 <code>
 inspect(dtm[1:2,1000:1005])
+inspect(dtm)
 </code>
@@ Line 108: / Line 109: @@
 <code>
 #create sort order (descending)
-ord <- order(freq,decreasing=TRUE)
+ord <- order(freq, decreasing=TRUE)
 </code>
@@ Line 120: / Line 121: @@
 <code>
+# word length 4 or more
 dtmr <-DocumentTermMatrix(docs, control=list(wordLengths=c(4, 20), bounds = list(global = c(3,27))))
 </code>
 <code>dtmr
+inspect(dtmr)
 </code>
@@ Line 165: / Line 168: @@
 set.seed(42)
 #limit words by specifying min frequency
-wordcloud(names(freqr),freqr, min.freq=70)
+wordcloud(names(freqr),freqr, min.freq=30)
 </code>
 <code>
 #…add color
-wordcloud(names(freqr),freqr,min.freq=70,colors=brewer.pal(6,"Dark2"))
+wordcloud(names(freqr),freqr,min.freq=30,colors=brewer.pal(6,"Dark2"))
 </code>
-====== e.g. 1 with output ======
+====== output ======
+<code>> library(tm)
+필요한 패키지를 로딩중입니다: NLP
-<code>> Sys.setlocale(category = "LC_ALL", locale = "US")
+다음의 패키지를 부착합니다: ‘NLP’
-[1] "LC_COLLATE=English_United States.1252;LC_CTYPE=English_United States.1252;LC_MONETARY=English_United States.1252;LC_NUMERIC=C;LC_TIME=English_United States.1252"
-> library(tm)
+The following object is masked from ‘package:ggplot2’:
+    annotate
+Warning messages:
+: 패키지 ‘tm’는 R 버전 3.4.4에서 작성되었습니다
+: 패키지 ‘NLP’는 R 버전 3.4.4에서 작성되었습니다
 > #Create Corpus
 > docs <- Corpus(DirSource("D:/Users/Hyo/Documents/TextMining"))
@@ Line 184: / Line 195: @@
 <<SimpleCorpus>>
 Metadata:  corpus specific: 1, document level (indexed): 0
-Content:  documents: 30
+Content:  documents: 31
 > #inspect a particular document
 > writeLines(as.character(docs[[30]]))
-Understanding "flexibility" ?a close-up view of an organizational platitude
+TOGAF or not TOGAF but is that the question?
+ "The Holy Grail of effective collaboration is creating shared understanding, which is a precursor to shared commitment." <96> Jeff Conklin.
+ "Without context, words and actions have no meaning at all." <96> Gregory Bateson.
+I spent much of last week attending a class on the TOGAF Enterprise Architecture (EA) framework.  Prior experience with  IT frameworks such as PMBOK and ITIL had taught me that much depends on the instructor <96> a good one can make the material come alive whereas a not-so-good one can make it an experience akin to watching grass grow. I neednt have worried: the instructor was superb, and my classmates, all of whom are experienced IT professionals / architects, livened up the proceedings through comments and discussions both in class and outside it. All in all, it was a thoroughly enjoyable and educative experience, something I cannot say for many of the professional courses I have attended.
+One of the things about that struck me about TOGAF is the way in which the components of the framework hang together to make a coherent whole (see the introductory chapter of the framework for an overview). To be sure, there is a lot of detail within those components, but there is a certain abstract elegance <96> dare I say, beauty <96> to the framework.
+That said TOGAF is (almost) entirely silent on the following question which I addressed in a post late last year:
+Why is Enterprise Architecture so hard to get right?
+Many answers have been offered. Here are some, extracted from articles published by IT vendors and consultancies:
-Introduction
+Lack of sponsorship
-Flexibility is one of those buzzwords that keeps coming up in organizational communiques and discussions. People are continually asked to display flexibility, without ever being told what the term means:  flexible workplaces, flexible attitudes, flexible jobs ?the word itself has a flexible meaning that depends on the context in which it is used and by whom.
+Not engaging the business
-When words are used in this way they become platitudes ?empty words that make a lot of noise. In this post, I analyse the platitude, flexibility, as it is used in organisations. My discussion is based on a paper by Thomas Eriksen entitled, Mind the Gap: Flexibility, Epistemology and the Rhetoric of New Work.
+Inadequate communication
-Background ?a bit about organizational platitudes
+Insensitivity to culture / policing mentality
-One of the things that struck me when I moved from academia to industry is the difference in the way words or phrases are used in the two domains. In academics one has to carefully define the terms one uses (particularly if one is coining a new term) whereas in business it doesn't seem to matter, words can mean whatever one wants them to mean (OK, this is an exaggeration, but not by too much). Indeed, as Paul Culmsee and I discuss in the first chapter of The Heretic's Guide to Best Practices, many terms that are commonly bandied about in organizations are platitudes because they are understood differently by different people.
+Clinging to a particular tool or framework
-A good example of a platitude is the word governance. One manager may see governance as being largely about oversight and control whereas another might interpret it as being about providing guidance.  Such varying interpretations can result in major differences in the way the two managers implement governance:  the first one might enforce it as a compliance-oriented set of processes that leave little room for individual judgement while the other might implement it as a broad set of guidelines that leave many of the decisions in the hands of those who are actually doing the work. Needless to say, the results in the two cases are likely to be different too.
+Building an ivory tower
-Flexibility ?the conventional view
+Wrong choice of architect
-A good place to start our discussion of flexibility is with the dictionary. The online Oxford Dictionary defines at as:
+(Note: the above points are taken from this article and this one)
-Flexibility (noun):
- the ability to be easily modified
+It is interesting that the first four issues listed are related to the fact that different stakeholders in an organization have vastly different perspectives on what an enterprise architecture initiative should achieve.  This lack of shared understanding is what makes enterprise architecture a socially complex problem rather than a technically difficult one. As Jeff Conklin points out in this article, problems that are technically complex will usually have a solution that will be acceptable to all stakeholders, whereas socially complex problems will not.  Sending a spacecraft to Mars is an example of the former whereas an organization-wide ERP  (or EA!) project or (on a global scale) climate change are instances of the latter.
- willingness to change or compromise
-The term is widely used in both these senses in organizational settings. For example, people speak of flexible designs (i.e. designs that can be easily modified) or flexible people (referring to those who are willing to change or compromise). However,  and this is the problem:  the term is open to interpretation ?what Jack might term a flexible approach may be seen by Jill as a complete lack of method. These differences in interpretation become particularly obvious when the word is used in a broad context ?such as in a statement justifying an organizational change.  An executive might see a corporate restructure and the resulting changes in jobs/roles as a means to achieve organizational flexibility, but those affected by it may see it as constraining theirs.  As Eriksen states:
+Interestingly, even the fifth and sixth points in the list above <96> framework dogma and retreating to an ivory tower <96> are usually consequences of the inability to manage social complexity. Indeed, that is precisely the point made in the final item in the list: enterprise architects are usually selected for their technical skills rather than their ability to deal with ambiguities that are characteristic of social complexity.
-Jobs are flexible in the sense that they are unstable and uncertain, few employees hold the same jobs for many years, the content of jobs can be changed almost overnight, and the boundaries between work and leisure are negotiable and chronically fuzzy.
-Indeed, such "flexibility" which requires one to change at short notice results in a fragmentation of individual experience and a resulting loss of a coherent narrative of one's life. It appears that increased flexibility in one aspect results in a loss of flexibility in another. Any sensible definition of flexibility ought to reflect this.
+TOGAF offers enterprise architects a wealth of tools to manage technical complexity. These need to be complemented by a suite of techniques to reconcile worldviews of different stakeholder groups.  Some examples of such techniques are Soft Systems Methodology, Polarity Management, and Dialogue Mapping. I wont go into details of these here, but if youre interested, please have a look at my posts entitled, The Approach <96> a dialogue mapping story and The dilemmas of enterprise IT for brief introductions to the latter two techniques via IT-based examples.
-Understanding flexibility
-Consider the following definition of flexibility proposed by Gregory Bateson:
+<Advertisement > Better yet, you could check out Chapter 9 of my book for a crash course on Soft Systems Methodology and Polarity Management and Dialogue Mapping, and the chapters thereafter for a deep dive into Dialogue Mapping </Advertisement>.
-"Flexibility is uncommitted potential for change"
-This deceptively simple statement is a good place to start understanding what flexibility really means for projects, organisations 꿢nd even software systems.
+Apart from social complexity, there is the problem of context <96> the circumstances that shape the unique culture and features of an organization.  As I mentioned in my introductory remarks, the framework is abstract <96> it applies to an ideal organization in which things can be done by the book. But such an organization does not exist!  Aside from unique people-related and political issues, all organisations have their own quirks and unique features that distinguish them from other organisations, even within the same domain. Despite superficial resemblances, no two pharmaceutical companies are alike. Indeed, the differences are the whole point because they are what make a particular organization what it is. To paraphrase the words of the anthropologist, Gregory Bateson, the differences are what make a difference.
-As Eriksen tells us, Bateson proposed this definition in the context of ecology. In particular, Bateson had in mind the now obvious notion that the increased flexibility we gain through our increasingly energy-hungry lifestyles results in a decrease in the environment's capacity to cope with the consequences. This is true of flexibility in any context: a gain in flexibility in one dimension will necessarily be accompanied by a loss of flexibility in another.
-Another implication of the above definition is that a system that is running at or near the limits of its operating variables cannot be flexible.  The following examples should make this clear:
+Some may argue that the framework acknowledges this and encourages, even exhorts, people to tailor the framework to their needs. Sure, the word  "tailor" and its variants appear almost 700 times in the version 9.1 of the standard but, once again, there is no advice offered on how this tailoring should be done.  And one can well understand why: it is impossible to offer any sensible advice if one doesnt know the specifics of the organization, which includes its context.
- A project team that is putting in 18 hour workdays in order to finish a project on time.
- A car that's being driven at top speed.
+On a related note, the TOGAF framework acknowledges that there is a hierarchy of architectures ranging from the general (foundation) to the specific (organization). However despite the acknowledgement of diversity,   in practice TOGAF tends to focus on similarities between organisations. Most of the prescribed building blocks and processes are based on assumed commonalities between the structures and processes in different organisations.   My point is that, although similarities are important, architects need to focus on differences. These could be differences between the organization they are working in and the TOGAF ideal, or even between their current organization and others that they have worked with in the past (and this is where experience comes in really handy). Cataloguing and understanding these unique features <96>  the differences that make a difference <96> draws attention to precisely those issues that can cause heartburn and sleepless nights later.
- A family living beyond their means.
-All these systems are operating at or near their limits, they have little or no spare capacity to accommodate change.
+I have often heard arguments along the lines of  "80% of what we do follows a standard process, so it should be easy for us to standardize on a framework." These are famous last words, because some of the 20% that is different is what makes your organization unique, and is therefore worthy of attention. You might as well accept this upfront so that you get a realistic picture of the challenges early in the game.
-A third implication of the definition follows from the preceding one:  the key variables of a flexible system should lie in the mid-range of their upper and lower limits. In terms of above examples:
- The project team should be putting in normal hours.
+To sum up, frameworks like TOGAF are abstractions based on an ideal organization; they gloss over social complexity and the unique context of individual organisations.  So, questions such as the one posed in the title of this post are akin to the pseudo-choice between Coke and Pepsi, for the real issue is something else altogether. As Tom Graves tells us in his wonderful blog and book, the enterprise is a story rather than a structure, and its architecture an ongoing sociotechnical drama.
- The car should be driven at or below the posted road speed limits
- The family should be living within its income, with a reasonable amount to spare.
-Of course, the whole point of ensuring that systems operate in their comfort zone is that they can be revved up if the need arises. Such revving up, however,  should be an exceptional circumstance rather than the norm ?a point that those who run projects, organisations (and, yes, even vehicles) often tend to forget. If one operates a system at the limits of its tolerance for too long, not only will it not be flexible, it will break.
-Flexibility in the workplace
-As mentioned in the introduction, the term flexibility keeps cropping up in organizational settings: corporate communiques exhort employees to be flexible in the face of change.  This is typically a coded signal that employees should expect uncertainty and be prepared to adjust to it.  A related manifestation of flexibility is the blurring of the distinction between work and personal life. As Eriksen puts it:
-The term flexibility is often used to describe this new situation: Jobs are flexible in the sense that they are unstable and uncertain, few employees hold the same jobs for many years, the content of jobs can be changed, and the boundaries between work and leisure are poorly defined.
-This trend is aided by recent developments in technology that enable employees to be perpetually on call. This is often sold as a work from home initiative but usually ends up being much more.  Eriksen has this to say about home offices:
-One recent innovation typically associated with flexibility is the home office. In Scandinavia (and some other prosperous, technologically optimistic regions), many companies equipped some of their employees with home computers with online access to the company network in the early 1990s, in order to enhance their flexibility. This was intended to enable employees to work from home part of the time, thereby making the era when office workers were chained to the office desk all day obsolete.
-In the early days, there were widespread worries among employers to the effect that a main outcome of this new flexibility would consist in a reduction of productivity. Since there was no legitimate way of checking how the staff actually spent their time out of the office, it was often suspected that they worked less from home than they were supposed to. If this were in fact the case, working from home would have led to a real increase in the flexibility of time budgeting. However, work researchers eventually came up with a different picture. By the late 1990s, hardly anybody spoke of the home office as a convenient way of escaping from work; rather, the concern among unionists as well as researchers was now that increasing numbers of employees were at pains to distinguish between working hours and leisure time, and were suffering symptoms of burnout and depression. The home office made it difficult to distinguish between contexts that were formerly mutually exclusive because of differ... <truncated>
-It is interesting to see this development in the light of Bateson's definition of flexibility: the employee gains flexibility in space (he or she can work from home or from the office) at the expense of flexibility in time(organization time encroaches on personal time). As Eriksen states:
- There seems to be a classic Batesonian flexibility trade-off associated with the new information technologies: increased spatial flexibility entails decreased temporal flexibility. If inaccessibility and 'empty time' are understood as scarce resources, the context of 'new work' thus seems to be an appropriate context for a new economics as well. In fact, a main environmental challenge of our near future will consist in protecting slow time and gaps from environmental degradation.
-In short, it appears that flexibility for the organization necessarily implies a loss of flexibility for the individual.
-Conclusion
-Flexibility is in the eye of the beholder: an action to increase organisational flexibility by, say, redeploying employees would likely be seen by those affected as a move that constrains their  (individual) flexibility.  Such a dual meaning is characteristic of many organizational platitudes such as Excellence, Synergy andGovernance. It is an interesting exercise to analyse such platitudes and expose the difference between their espoused and actual meanings.  So I sign off for 2013, wishing you  many hours of platitude-deconstructing fun :-)
 > getTransformations()
-[1] "removeNumbers"     "removePunctuation" "removeWords"       "stemDocument"      "stripWhitespace"
+[1] "removeNumbers"     "removePunctuation" "removeWords"
+[4] "stemDocument"      "stripWhitespace"
 > #create the toSpace content transformer
 > toSpace <- content_transformer(function(x, pattern) {return (gsub(pattern, " ", x))})
@@ Line 250: / Line 262: @@
 > docs <- tm_map(docs, stripWhitespace)
 > writeLines(as.character(docs[[30]]))
-understanding flexibility ?close view organizational platitude introduction flexibility one buzzwords keeps coming organizational communiques discussions people continually asked display flexibility without ever told term means flexible workplaces flexible attitudes flexible jobs ?word flexible meaning depends context used words used way become platitudes ?empty words make lot noise post analyse platitude flexibility used organisations discussion based paper thomas eriksen entitled mind gap flexibility epistemology rhetoric new work background ?bit organizational platitudes one things struck moved academia industry difference way words phrases used two domains academics one carefully define terms one uses particularly one coining new term whereas business doesnt seem matter words can mean whatever one wants mean ok exaggeration much indeed paul culmsee discuss first chapter heretics guide best practices many terms commonly bandied organizations platitudes understood differently differe... <truncated>
+togaf togaf question holy grail effective collaboration creating shared understanding precursor shared commitment jeff conklin without context words actions meaning gregory bateson spent much last week attending class togaf enterprise architecture ea framework prior experience frameworks pmbok itil taught much depends instructor good one can make material come alive whereas good one can make experience akin watching grass grow neednt worried instructor superb classmates experienced professionals architects livened proceedings comments discussions class outside thoroughly enjoyable educative experience something say many professional courses attended one things struck togaf way components framework hang together make coherent whole see introductory chapter framework overview sure lot detail within components certain abstract elegance dare say beauty framework said togaf almost entirely silent following question addressed post late last year enterprise architecture hard get right many an... <truncated>
 > #load library
 > library(SnowballC)
+Warning message:
+패키지 ‘SnowballC’는 R 버전 3.4.4에서 작성되었습니다
 >
 > #Stem document
 > docs <- tm_map(docs,stemDocument)
 > writeLines(as.character(docs[[30]]))
-understand flexibl ?close view organiz platitud introduct flexibl one buzzword keep come organiz communiqu discuss peopl continu ask display flexibl without ever told term mean flexibl workplac flexibl attitud flexibl job ?word flexibl mean depend context use word use way becom platitud ?empti word make lot nois post analys platitud flexibl use organis discuss base paper thoma eriksen entitl mind gap flexibl epistemolog rhetor new work background ?bit organiz platitud one thing struck move academia industri differ way word phrase use two domain academ one care defin term one use particular one coin new term wherea busi doesnt seem matter word can mean whatev one want mean ok exagger much inde paul culmse discuss first chapter heret guid best practic mani term common bandi organ platitud understood differ differ peopl good exampl platitud word govern one manag may see govern larg oversight control wherea anoth might interpret provid guidanc vari interpret can result major differ way two... <truncated>
+togaf togaf question holi grail effect collabor creat share understand precursor share commit jeff conklin without context word action mean gregori bateson spent much last week attend class togaf enterpris architectur ea framework prior experi framework pmbok itil taught much depend instructor good one can make materi come aliv wherea good one can make experi akin watch grass grow neednt worri instructor superb classmat experienc profession architect liven proceed comment discuss class outsid thorough enjoy educ experi someth say mani profession cours attend one thing struck togaf way compon framework hang togeth make coher whole see introductori chapter framework overview sure lot detail within compon certain abstract eleg dare say beauti framework said togaf almost entir silent follow question address post late last year enterpris architectur hard get right mani answer offer extract articl publish vendor consult lack sponsorship engag busi inadequ communic insensit cultur polic menta... <truncated>
 > docs <- tm_map(docs, content_transformer(gsub), pattern = "organiz", replacement = "organ")
 > docs <- tm_map(docs, content_transformer(gsub), pattern = "organis", replacement = "organ")
@@ Line 265: / Line 279: @@
 > dtm <- DocumentTermMatrix(docs)
 > dtm
-<<DocumentTermMatrix (documents: 30, terms: 3902)>>
+<<DocumentTermMatrix (documents: 31, terms: 3892)>>
-Non-/sparse entries: 13979/103081
+Non-/sparse entries: 13977/106675
 Sparsity           : 88%
-Maximal term length: 48
+Maximal term length: 54
 Weighting          : term frequency (tf)
 > inspect(dtm[1:2,1000:1005])
@@ Line 274: / Line 288: @@
 Non-/sparse entries: 0/12
 Sparsity           : 100%
-Maximal term length: 7
+Maximal term length: 12
 Weighting          : term frequency (tf)
 Sample             :
                                     Terms
-Docs                                 critic current cya cynefin david decid
+Docs                                 decid decis defineeffici
-  BeyondEntitiesAndRelationships.txt      0       0   0       0     0     0
+  BeyondEntitiesAndRelationships.txt     0     0            0
-  bigdata.txt                             0       0   0       0     0     0
+  bigdata.txt                            0     0            0
+                                    Terms
+Docs                                 degre demand devoid
+  BeyondEntitiesAndRelationships.txt     0      0      0
+  bigdata.txt                            0      0      0
+> inspect(dtm)
+<<DocumentTermMatrix (documents: 31, terms: 3892)>>
+Non-/sparse entries: 13977/106675
+Sparsity           : 88%
+Maximal term length: 54
+Weighting          : term frequency (tf)
+Sample             :
+                                       Terms
+Docs                                    can manag one organ
+  BeyondEntitiesAndRelationships.txt     26     8  15     8
+  ConditionsOverCauses.txt                8     9   7    14
+  EmergentDesignInEnterpriseIT.txt       15     6  28     8
+  FromInformationToKnowledge.txt         35     7  17     9
+  MakingSenseOfOrganizationalChange.txt  15    10  26    27
+  MakingSenseOfSensemaking.txt           25     7  26    12
+  RoutinesAndReality.txt                  8     3  13    10
+  SixHeresiesForBI.txt                    6     3  10     7
+  TheEssenceOfEntrepreneurship.txt        5     2  24     2
+  ThreeTypesOfUncertainty.txt            13     9  18     3
+                                       Terms
+Docs                                    problem project system
+  BeyondEntitiesAndRelationships.txt          5       1      6
+  ConditionsOverCauses.txt                    5       2      4
+  EmergentDesignInEnterpriseIT.txt           16      17     13
+  FromInformationToKnowledge.txt             16       4     21
+  MakingSenseOfOrganizationalChange.txt      15       7      6
+  MakingSenseOfSensemaking.txt               12      19      9
+  RoutinesAndReality.txt                      6       4     36
+  SixHeresiesForBI.txt                        4       0      4
+  TheEssenceOfEntrepreneurship.txt            5       1      0
+  ThreeTypesOfUncertainty.txt                15       1      0
+                                       Terms
+Docs                                    use way work
+  BeyondEntitiesAndRelationships.txt     18   9    0
+  ConditionsOverCauses.txt                1   3   13
+  EmergentDesignInEnterpriseIT.txt       14  11    4
+  FromInformationToKnowledge.txt         25   8   10
+  MakingSenseOfOrganizationalChange.txt  10  27   25
+  MakingSenseOfSensemaking.txt           27  22   13
+  RoutinesAndReality.txt                 16   9   13
+  SixHeresiesForBI.txt                    9   2    5
+  TheEssenceOfEntrepreneurship.txt        6  30   10
+  ThreeTypesOfUncertainty.txt             3   4    7
 > freq <- colSums(as.matrix(dtm))
 > #length should be total number of terms
 > length(freq)
-[1] 3902
+[1] 3892
 > #create sort order (descending)
-> ord <- order(freq,decreasing=TRUE)
+> ord <- order(freq, decreasing=TRUE)
 > #inspect most frequently occurring terms
 > freq[head(ord)]
    one  organ    can  manag   work system
-    275    244    230    209    193
+    276    244    230    210    193
 >
 > #inspect least frequently occurring terms
@@ Line 296: / Line 357: @@
   therebi timeorgan  uncommit  unionist   willing   workday
          1         1         1         1         1
+> # word length 4 or more
 > dtmr <-DocumentTermMatrix(docs, control=list(wordLengths=c(4, 20), bounds = list(global = c(3,27))))
 > dtmr
-<<DocumentTermMatrix (documents: 30, terms: 1294)>>
+<<DocumentTermMatrix (documents: 31, terms: 1295)>>
-Non-/sparse entries: 10071/28749
+Non-/sparse entries: 10082/30063
-Sparsity           : 74%
+Sparsity           : 75%
 Maximal term length: 15
 Weighting          : term frequency (tf)
-> freqr <- colSums(as.matrix(dtmr))
-> #length should be total number of terms
-> length(freqr)
-[1] 1294
->
-> #create sort order (asc)
-> ordr <- order(freqr,decreasing=TRUE)
->
-> #inspect most frequently occurring terms
-> freqr[head(ordr)]
-  organ   manag    work  system project problem
-     230     209     193     185     173
->
-> #inspect least frequently occurring terms
-> freqr[tail(ordr)]
-  hmmm struck multin  lower pseudo  gloss
-      3      3      3      3      3
-> findFreqTerms(dtmr,lowfreq=80)
- [1] "action"     "approach"   "base"       "busi"       "data"       "design"     "develop"    "differ"     "discuss"    "enterpris"  "exampl"
-[12] "group"      "howev"      "import"     "issu"       "make"       "manag"      "mani"       "model"      "often"      "organ"      "peopl"
-[23] "point"      "practic"    "problem"    "process"    "project"    "question"   "said"       "situat"     "system"     "thing"      "think"
-[34] "time"       "understand" "view"       "well"       "will"       "work"       "chang"      "consult"    "decis"      "even"       "like"
-> findAssocs(dtmr, "project", 0.6)
-$project
- inher  manag  handl occurr
-.82   0.69   0.68   0.67
-> findAssocs(dtmr, "enterpris", 0.6)
-$enterpris
-       agil   increment     realist     upfront   technolog        solv     neither    movement       happi       adapt   architect architectur       chanc
-.81        0.79        0.77        0.76        0.69        0.68        0.68        0.66        0.66        0.65        0.65        0.65        0.63
-       fine      featur
-.63        0.62
-> findAssocs(dtmr, "system", 0.6)
-$system
-  design   subset    adopt     user   involv  specifi function   intend     step  softwar   specif   intent   compos   depart    phone frequent    today
-.78     0.78     0.77     0.75     0.71     0.71     0.70     0.67     0.67     0.67     0.66     0.66     0.66     0.65     0.63     0.62     0.62
- pattern   author   wherea   cognit
-.61     0.60     0.60     0.60
-> wf=data.frame(term=names(freqr),occurrences=freqr)
-> library(ggplot2)
-> p <- ggplot(subset(wf, freqr>100), aes(term, occurrences))
-> p <- p + geom_bar(stat="identity")
-> p <- p + theme(axis.text.x=element_text(angle=45, hjust=1))
-> p
-> #wordcloud
-> library(wordcloud)
-> #setting the same seed each time ensures consistent look across clouds
-> set.seed(42)
-> #limit words by specifying min frequency
-> wordcloud(names(freqr),freqr, min.freq=70)
-> #…add color
-> wordcloud(names(freqr),freqr,min.freq=70,colors=brewer.pal(6,"Dark2"))
-> dtmr
-<<DocumentTermMatrix (documents: 30, terms: 1294)>>
-Non-/sparse entries: 10071/28749
-Sparsity           : 74%
-Maximal term length: 15
-Weighting          : term frequency (tf)
-> inspect(dtmr[1:5])
-Error in x$nrow : $ operator is invalid for atomic vectors
 > inspect(dtmr)
-<<DocumentTermMatrix (documents: 30, terms: 1294)>>
+<<DocumentTermMatrix (documents: 31, terms: 1295)>>
-Non-/sparse entries: 10071/28749
+Non-/sparse entries: 10082/30063
-Sparsity           : 74%
+Sparsity           : 75%
 Maximal term length: 15
 Weighting          : term frequency (tf)
 Sample             :
                                        Terms
-Docs                                    approach differ exampl manag organ problem project system will work
+Docs                                    differ exampl manag
-  BeyondEntitiesAndRelationships.txt          13     14      6     8     8       5       1      6   12    0
+  BeyondEntitiesAndRelationships.txt        14      6     8
-  ConditionsOverCauses.txt                    16      2      5     9    14       5       2      4    5   13
+  ConditionsOverCauses.txt                   2      5     9
-  EmergentDesignInEnterpriseIT.txt            13     11      8     6     8      16      17     13   11    4
+  EmergentDesignInEnterpriseIT.txt          11      8     6
-  FromInformationToKnowledge.txt               2      7     21     7     9      16       4     21    5   10
+  FromInformationToKnowledge.txt             7     21     7
-  MakingSenseOfOrganizationalChange.txt        7      4     10    10    26      14       7      6   12   25
+  MakingSenseOfOrganizationalChange.txt      4     10    10
-  MakingSenseOfSensemaking.txt                13     17     15     7    12      12      16      9   18   12
+  MakingSenseOfSensemaking.txt              17     15     7
-  RoutinesAndReality.txt                       1      7      2     3    10       6       4     36    8   13
+  RoutinesAndReality.txt                     7      2     3
-  SixHeresiesForBI.txt                         2      2      7     3     7       4       0      4    6    5
+  SixHeresiesForBI.txt                       2      7     3
-  TheEssenceOfEntrepreneurship.txt             7     15     15     2     2       5       1      0   10   10
+  TheEssenceOfEntrepreneurship.txt          15     15     2
-  ThreeTypesOfUncertainty.txt                 20      6     15     9     3      15       1      0    6    7
+  ThreeTypesOfUncertainty.txt                6     15     9
-> dtm
-<<DocumentTermMatrix (documents: 30, terms: 3902)>>
-Non-/sparse entries: 13979/103081
-Sparsity           : 88%
-Maximal term length: 48
-Weighting          : term frequency (tf)
-> dtmr
-<<DocumentTermMatrix (documents: 30, terms: 1294)>>
-Non-/sparse entries: 10071/28749
-Sparsity           : 74%
-Maximal term length: 15
-Weighting          : term frequency (tf)
-> inspect(dtm)
-<<DocumentTermMatrix (documents: 30, terms: 3902)>>
-Non-/sparse entries: 13979/103081
-Sparsity           : 88%
-Maximal term length: 48
-Weighting          : term frequency (tf)
-Sample             :
                                        Terms
-Docs                                    can manag one organ problem project system use way work
+Docs                                    organ problem project
-  BeyondEntitiesAndRelationships.txt     26     8  15     8       5       1      6  18   9    0
+  BeyondEntitiesAndRelationships.txt        8       5       1
-  ConditionsOverCauses.txt                8     9   7    14       5       2      4   1   3   13
+  ConditionsOverCauses.txt                 14       5       2
-  EmergentDesignInEnterpriseIT.txt       15     6  28     8      16      17     13  14  11    4
+  EmergentDesignInEnterpriseIT.txt          8      16      17
-  FromInformationToKnowledge.txt         35     7  17     9      16       4     21  25   8   10
+  FromInformationToKnowledge.txt            9      16       4
-  MakingSenseOfOrganizationalChange.txt  15    10  26    26      14       7      6  10  27   25
+  MakingSenseOfOrganizationalChange.txt    27      15       7
-  MakingSenseOfSensemaking.txt           25     7  26    12      12      16      9  27  21   12
+  MakingSenseOfSensemaking.txt             12      12      19
-  RoutinesAndReality.txt                  8     3  13    10       6       4     36  16   9   13
+  RoutinesAndReality.txt                   10       6       4
-  SixHeresiesForBI.txt                    6     3  10     7       4       0      4   9   2    5
+  SixHeresiesForBI.txt                      7       4       0
-  TheEssenceOfEntrepreneurship.txt        5     2  24     2       5       1      0   6  30   10
+  TheEssenceOfEntrepreneurship.txt          2       5       1
-  ThreeTypesOfUncertainty.txt            13     9  18     3      15       1      0   3   4    7
+  ThreeTypesOfUncertainty.txt               3      15       1
-> Sys.setlocale(category = "LC_ALL", locale = "US")
+                                       Terms
-[1] "LC_COLLATE=English_United States.1252;LC_CTYPE=English_United States.1252;LC_MONETARY=English_United States.1252;LC_NUMERIC=C;LC_TIME=English_United States.1252"
+Docs                                    question system will
-> library(tm)
+  BeyondEntitiesAndRelationships.txt          19      6   12
-> #Create Corpus
+  ConditionsOverCauses.txt                     0      4    5
-> docs <- Corpus(DirSource("D:/Users/Hyo/Documents/TextMining"))
+  EmergentDesignInEnterpriseIT.txt             8     13   11
-> docs
+  FromInformationToKnowledge.txt              16     21    5
-<<SimpleCorpus>>
+  MakingSenseOfOrganizationalChange.txt       10      6   12
-Metadata:  corpus specific: 1, document level (indexed): 0
+  MakingSenseOfSensemaking.txt                63      9   18
-Content:  documents: 30
+  RoutinesAndReality.txt                       2     36    8
-> #inspect a particular document
+  SixHeresiesForBI.txt                         4      4    6
-> writeLines(as.character(docs[[30]]))
+  TheEssenceOfEntrepreneurship.txt             0      0   10
-Understanding "flexibility" ?a close-up view of an organizational platitude
+  ThreeTypesOfUncertainty.txt                  2      0    6
+                                       Terms
-Introduction
+Docs                                    work
-Flexibility is one of those buzzwords that keeps coming up in organizational communiques and discussions. People are continually asked to display flexibility, without ever being told what the term means:  flexible workplaces, flexible attitudes, flexible jobs ?the word itself has a flexible meaning that depends on the context in which it is used and by whom.
+  BeyondEntitiesAndRelationships.txt       0
-When words are used in this way they become platitudes ?empty words that make a lot of noise. In this post, I analyse the platitude, flexibility, as it is used in organisations. My discussion is based on a paper by Thomas Eriksen entitled, Mind the Gap: Flexibility, Epistemology and the Rhetoric of New Work.
+  ConditionsOverCauses.txt                13
-Background ?a bit about organizational platitudes
+  EmergentDesignInEnterpriseIT.txt         4
-One of the things that struck me when I moved from academia to industry is the difference in the way words or phrases are used in the two domains. In academics one has to carefully define the terms one uses (particularly if one is coining a new term) whereas in business it doesn't seem to matter, words can mean whatever one wants them to mean (OK, this is an exaggeration, but not by too much). Indeed, as Paul Culmsee and I discuss in the first chapter of The Heretic's Guide to Best Practices, many terms that are commonly bandied about in organizations are platitudes because they are understood differently by different people.
+  FromInformationToKnowledge.txt          10
-A good example of a platitude is the word governance. One manager may see governance as being largely about oversight and control whereas another might interpret it as being about providing guidance.  Such varying interpretations can result in major differences in the way the two managers implement governance:  the first one might enforce it as a compliance-oriented set of processes that leave little room for individual judgement while the other might implement it as a broad set of guidelines that leave many of the decisions in the hands of those who are actually doing the work. Needless to say, the results in the two cases are likely to be different too.
+  MakingSenseOfOrganizationalChange.txt   25
-Flexibility ?the conventional view
+  MakingSenseOfSensemaking.txt            13
-A good place to start our discussion of flexibility is with the dictionary. The online Oxford Dictionary defines at as:
+  RoutinesAndReality.txt                  13
-Flexibility (noun):
+  SixHeresiesForBI.txt                     5
- the ability to be easily modified
+  TheEssenceOfEntrepreneurship.txt        10
- willingness to change or compromise
+  ThreeTypesOfUncertainty.txt              7
-The term is widely used in both these senses in organizational settings. For example, people speak of flexible designs (i.e. designs that can be easily modified) or flexible people (referring to those who are willing to change or compromise). However,  and this is the problem:  the term is open to interpretation ?what Jack might term a flexible approach may be seen by Jill as a complete lack of method. These differences in interpretation become particularly obvious when the word is used in a broad context ?such as in a statement justifying an organizational change.  An executive might see a corporate restructure and the resulting changes in jobs/roles as a means to achieve organizational flexibility, but those affected by it may see it as constraining theirs.  As Eriksen states:
-Jobs are flexible in the sense that they are unstable and uncertain, few employees hold the same jobs for many years, the content of jobs can be changed almost overnight, and the boundaries between work and leisure are negotiable and chronically fuzzy.
-Indeed, such "flexibility" which requires one to change at short notice results in a fragmentation of individual experience and a resulting loss of a coherent narrative of one's life. It appears that increased flexibility in one aspect results in a loss of flexibility in another. Any sensible definition of flexibility ought to reflect this.
-Understanding flexibility
-Consider the following definition of flexibility proposed by Gregory Bateson:
-"Flexibility is uncommitted potential for change"
-This deceptively simple statement is a good place to start understanding what flexibility really means for projects, organisations 꿢nd even software systems.
-As Eriksen tells us, Bateson proposed this definition in the context of ecology. In particular, Bateson had in mind the now obvious notion that the increased flexibility we gain through our increasingly energy-hungry lifestyles results in a decrease in the environment's capacity to cope with the consequences. This is true of flexibility in any context: a gain in flexibility in one dimension will necessarily be accompanied by a loss of flexibility in another.
-Another implication of the above definition is that a system that is running at or near the limits of its operating variables cannot be flexible.  The following examples should make this clear:
- A project team that is putting in 18 hour workdays in order to finish a project on time.
- A car that's being driven at top speed.
- A family living beyond their means.
-All these systems are operating at or near their limits, they have little or no spare capacity to accommodate change.
-A third implication of the definition follows from the preceding one:  the key variables of a flexible system should lie in the mid-range of their upper and lower limits. In terms of above examples:
- The project team should be putting in normal hours.
- The car should be driven at or below the posted road speed limits
- The family should be living within its income, with a reasonable amount to spare.
-Of course, the whole point of ensuring that systems operate in their comfort zone is that they can be revved up if the need arises. Such revving up, however,  should be an exceptional circumstance rather than the norm ?a point that those who run projects, organisations (and, yes, even vehicles) often tend to forget. If one operates a system at the limits of its tolerance for too long, not only will it not be flexible, it will break.
-Flexibility in the workplace
-As mentioned in the introduction, the term flexibility keeps cropping up in organizational settings: corporate communiques exhort employees to be flexible in the face of change.  This is typically a coded signal that employees should expect uncertainty and be prepared to adjust to it.  A related manifestation of flexibility is the blurring of the distinction between work and personal life. As Eriksen puts it:
-The term flexibility is often used to describe this new situation: Jobs are flexible in the sense that they are unstable and uncertain, few employees hold the same jobs for many years, the content of jobs can be changed, and the boundaries between work and leisure are poorly defined.
-This trend is aided by recent developments in technology that enable employees to be perpetually on call. This is often sold as a work from home initiative but usually ends up being much more.  Eriksen has this to say about home offices:
-One recent innovation typically associated with flexibility is the home office. In Scandinavia (and some other prosperous, technologically optimistic regions), many companies equipped some of their employees with home computers with online access to the company network in the early 1990s, in order to enhance their flexibility. This was intended to enable employees to work from home part of the time, thereby making the era when office workers were chained to the office desk all day obsolete.
-In the early days, there were widespread worries among employers to the effect that a main outcome of this new flexibility would consist in a reduction of productivity. Since there was no legitimate way of checking how the staff actually spent their time out of the office, it was often suspected that they worked less from home than they were supposed to. If this were in fact the case, working from home would have led to a real increase in the flexibility of time budgeting. However, work researchers eventually came up with a different picture. By the late 1990s, hardly anybody spoke of the home office as a convenient way of escaping from work; rather, the concern among unionists as well as researchers was now that increasing numbers of employees were at pains to distinguish between working hours and leisure time, and were suffering symptoms of burnout and depression. The home office made it difficult to distinguish between contexts that were formerly mutually exclusive because of differ... <truncated>
-It is interesting to see this development in the light of Bateson's definition of flexibility: the employee gains flexibility in space (he or she can work from home or from the office) at the expense of flexibility in time(organization time encroaches on personal time). As Eriksen states:
- There seems to be a classic Batesonian flexibility trade-off associated with the new information technologies: increased spatial flexibility entails decreased temporal flexibility. If inaccessibility and 'empty time' are understood as scarce resources, the context of 'new work' thus seems to be an appropriate context for a new economics as well. In fact, a main environmental challenge of our near future will consist in protecting slow time and gaps from environmental degradation.
-In short, it appears that flexibility for the organization necessarily implies a loss of flexibility for the individual.
-Conclusion
-Flexibility is in the eye of the beholder: an action to increase organisational flexibility by, say, redeploying employees would likely be seen by those affected as a move that constrains their  (individual) flexibility.  Such a dual meaning is characteristic of many organizational platitudes such as Excellence, Synergy andGovernance. It is an interesting exercise to analyse such platitudes and expose the difference between their espoused and actual meanings.  So I sign off for 2013, wishing you  many hours of platitude-deconstructing fun :-)
-> getTransformations()
-[1] "removeNumbers"     "removePunctuation" "removeWords"       "stemDocument"
-[5] "stripWhitespace"
-> #create the toSpace content transformer
-> toSpace <- content_transformer(function(x, pattern) {return (gsub(pattern, " ", x))})
-> docs <- tm_map(docs, toSpace, "-")
-> docs <- tm_map(docs, toSpace, ":")
-> #Remove punctuation ? replace punctuation marks with " "
-> docs <- tm_map(docs, removePunctuation)
->
-> docs <- tm_map(docs, toSpace, "’")
-> docs <- tm_map(docs, toSpace, "‘")
-> docs <- tm_map(docs, toSpace, " -")
-> #Transform to lower case (need to wrap in content_transformer)
-> docs <- tm_map(docs,content_transformer(tolower))
-> #Strip digits (std transformation, so no need for content_transformer)
-> docs <- tm_map(docs, removeNumbers)
-> #remove stopwords using the standard list in tm
-> docs <- tm_map(docs, removeWords, stopwords("english"))
-> #Strip whitespace (cosmetic?)
-> docs <- tm_map(docs, stripWhitespace)
-> writeLines(as.character(docs[[30]]))
-understanding flexibility ?close view organizational platitude introduction flexibility one buzzwords keeps coming organizational communiques discussions people continually asked display flexibility without ever told term means flexible workplaces flexible attitudes flexible jobs ?word flexible meaning depends context used words used way become platitudes ?empty words make lot noise post analyse platitude flexibility used organisations discussion based paper thomas eriksen entitled mind gap flexibility epistemology rhetoric new work background ?bit organizational platitudes one things struck moved academia industry difference way words phrases used two domains academics one carefully define terms one uses particularly one coining new term whereas business doesnt seem matter words can mean whatever one wants mean ok exaggeration much indeed paul culmsee discuss first chapter heretics guide best practices many terms commonly bandied organizations platitudes understood differently differe... <truncated>
-> #load library
-> library(SnowballC)
->
-> #Stem document
-> docs <- tm_map(docs,stemDocument)
-> writeLines(as.character(docs[[30]]))
-understand flexibl ?close view organiz platitud introduct flexibl one buzzword keep come organiz communiqu discuss peopl continu ask display flexibl without ever told term mean flexibl workplac flexibl attitud flexibl job ?word flexibl mean depend context use word use way becom platitud ?empti word make lot nois post analys platitud flexibl use organis discuss base paper thoma eriksen entitl mind gap flexibl epistemolog rhetor new work background ?bit organiz platitud one thing struck move academia industri differ way word phrase use two domain academ one care defin term one use particular one coin new term wherea busi doesnt seem matter word can mean whatev one want mean ok exagger much inde paul culmse discuss first chapter heret guid best practic mani term common bandi organ platitud understood differ differ peopl good exampl platitud word govern one manag may see govern larg oversight control wherea anoth might interpret provid guidanc vari interpret can result major differ way two... <truncated>
-> docs <- tm_map(docs, content_transformer(gsub), pattern = "organiz", replacement = "organ")
-> docs <- tm_map(docs, content_transformer(gsub), pattern = "organis", replacement = "organ")
-> docs <- tm_map(docs, content_transformer(gsub), pattern = "andgovern", replacement = "govern")
-> docs <- tm_map(docs, content_transformer(gsub), pattern = "inenterpris", replacement = "enterpris")
-> docs <- tm_map(docs, content_transformer(gsub), pattern = "team-", replacement = "team")
-> dtm <- DocumentTermMatrix(docs)
-> dtm
-<<DocumentTermMatrix (documents: 30, terms: 3902)>>
-Non-/sparse entries: 13979/103081
-Sparsity           : 88%
-Maximal term length: 48
-Weighting          : term frequency (tf)
-> inspect(dtm[1:2,1000:1005])
-<<DocumentTermMatrix (documents: 2, terms: 6)>>
-Non-/sparse entries: 0/12
-Sparsity           : 100%
-Maximal term length: 7
-Weighting          : term frequency (tf)
-Sample             :
-                                    Terms
-Docs                                 critic current cya cynefin david decid
-  BeyondEntitiesAndRelationships.txt      0       0   0       0     0     0
-  bigdata.txt                             0       0   0       0     0     0
-> freq <- colSums(as.matrix(dtm))
-> #length should be total number of terms
-> length(freq)
-[1] 3902
-> #create sort order (descending)
-> ord <- order(freq,decreasing=TRUE)
-> #inspect most frequently occurring terms
-> freq[head(ord)]
-   one  organ    can  manag   work system
-    275    244    230    209    193
->
-> #inspect least frequently occurring terms
-> freq[tail(ord)]
-  therebi timeorgan  uncommit  unionist   willing   workday
-         1         1         1         1         1
-> dtmr <-DocumentTermMatrix(docs, control=list(wordLengths=c(4, 20), bounds = list(global = c(3,27))))
-> dtmr
-<<DocumentTermMatrix (documents: 30, terms: 1294)>>
-Non-/sparse entries: 10071/28749
-Sparsity           : 74%
-Maximal term length: 15
-Weighting          : term frequency (tf)
 > freqr <- colSums(as.matrix(dtmr))
 > #length should be total number of terms
 > length(freqr)
-[1] 1294
+[1] 1295
 >
 > #create sort order (asc)
@@ Line 555: / Line 431: @@
 > freqr[head(ordr)]
   organ   manag    work  system project problem
-     230     209     193     185     173
+     230     210     193     188     174
 >
 > #inspect least frequently occurring terms
@@ Line 562: / Line 438: @@
       3      3      3      3      3
 > findFreqTerms(dtmr,lowfreq=80)
- [1] "action"     "approach"   "base"       "busi"       "data"       "design"
+ [1] "action"     "approach"   "base"       "busi"
- [7] "develop"    "differ"     "discuss"    "enterpris"  "exampl"     "group"
+ [5] "data"       "design"     "develop"    "differ"
-[13] "howev"      "import"     "issu"       "make"       "manag"      "mani"
+ [9] "discuss"    "enterpris"  "exampl"     "group"
-[19] "model"      "often"      "organ"      "peopl"      "point"      "practic"
+[13] "howev"      "import"     "issu"       "make"
-[25] "problem"    "process"    "project"    "question"   "said"       "situat"
+[17] "manag"      "mani"       "model"      "often"
-[31] "system"     "thing"      "think"      "time"       "understand" "view"
+[21] "organ"      "peopl"      "point"      "practic"
-[37] "well"       "will"       "work"       "chang"      "consult"    "decis"
+[25] "problem"    "process"    "project"    "question"
-[43] "even"       "like"
+[29] "said"       "situat"     "system"     "thing"
+[33] "think"      "time"       "understand" "view"
+[37] "well"       "will"       "work"       "chang"
+[41] "consult"    "decis"      "even"       "like"
 > findAssocs(dtmr, "project", 0.6)
 $project
- inher  manag  handl occurr
+    inher     manag    occurr     handl experienc
-.82   0.69   0.68   0.67
+.80      0.68      0.66      0.66      0.60
 > findAssocs(dtmr, "enterpris", 0.6)
 $enterpris
-       agil   increment     realist     upfront   technolog        solv     neither
+       agil   increment     realist     upfront   technolog
-.81        0.79        0.77        0.76        0.69        0.68        0.68
+.81        0.79        0.77        0.76        0.69
-   movement       happi       adapt   architect architectur       chanc        fine
+       solv     neither    movement       adapt   architect
-.66        0.66        0.65        0.65        0.65        0.63        0.63
+.68        0.68        0.66        0.66        0.65
-     featur
+architectur       chanc        fine      featur
-.62
+.65        0.64        0.64        0.62
 > findAssocs(dtmr, "system", 0.6)
 $system
-  design   subset    adopt     user   involv  specifi function   intend     step  softwar
+  design   subset    adopt     user   involv  specifi function
-.78     0.78     0.77     0.75     0.71     0.71     0.70     0.67     0.67     0.67
+.78     0.78     0.77     0.75     0.72     0.71     0.70
-  specif   intent   compos   depart    phone frequent    today  pattern   author   wherea
+  intend     step  softwar   specif   intent   compos   depart
-.66     0.66     0.66     0.65     0.63     0.62     0.62     0.61     0.60     0.60
+.68     0.68     0.68     0.67     0.66     0.66     0.65
-  cognit
+   phone frequent   author   wherea  pattern   cognit
-.60
+.64     0.63     0.61     0.61     0.61     0.60
 > wf=data.frame(term=names(freqr),occurrences=freqr)
@@ Line 601: / Line 480: @@
 > #wordcloud
 > library(wordcloud)
+필요한 패키지를 로딩중입니다: RColorBrewer
+Warning messages:
+: 패키지 ‘wordcloud’는 R 버전 3.4.4에서 작성되었습니다
+: 패키지 ‘RColorBrewer’는 R 버전 3.4.4에서 작성되었습니다
 > #setting the same seed each time ensures consistent look across clouds
 > set.seed(42)
 > #limit words by specifying min frequency
-> wordcloud(names(freqr),freqr, min.freq=70)
+> wordcloud(names(freqr),freqr, min.freq=30)
+There were 50 or more warnings (use warnings() to see the first 50)
 > #…add color
-> wordcloud(names(freqr),freqr,min.freq=70,colors=brewer.pal(6,"Dark2"))
+> wordcloud(names(freqr),freqr,min.freq=30,colors=brewer.pal(6,"Dark2"))
+There were 50 or more warnings (use warnings() to see the first 50)
+>
 </code>
+{{:c:itamc:pasted:20191112-151257.png}}
+{{:c:itamc:pasted:20191112-151307.png}}