This time, We survived the AWS outage

October 25, 2012 / / 1 Comment

Another minor bump

Anyone based in the US East region in AWS knows that yet again there were issues with EBS volumes, although you wouldn’t know it if you looked at their website. It’s a bit of a joke when you see headlines like Amazon outage takes down Reddit, Foursquare, others yet on their status page a tiny little note icon appears that states there’s a slight issue, extremely minor, don’t worry about it. Yeah right.

The main culprits were EC2 and the API, both of which were EBS related.

“Degraded EBS performance in a single Availability Zone
10:38 AM PDT We are currently investigating degraded performance for a small number of EBS volumes in a single Availability Zone in the US-EAST-1 Region.
11:11 AM PDT We can confirm degraded performance for a small number of EBS volumes in a single Availability Zone in the US-EAST-1 Region. Instances using affected EBS volumes will also experience degraded performance.
11:26 AM PDT We are currently experiencing degraded performance for EBS volumes in a single Availability Zone in the US-EAST-1 Region. New launches for EBS backed instances are failing and instances using affected EBS volumes will experience degraded performance.
12:32 PM PDT We are working on recovering the impacted EBS volumes in a single Availability Zone in the US-EAST-1 Region.
1:02 PM PDT We continue to work to resolve the issue affecting EBS volumes in a single availability zone in the US-EAST-1 region. The AWS Management Console for EC2 indicates which availability zone is impaired. “

The actual message is much much longer but you get the gist, a small number of people were affected. Yet most of the major websites that use amazon were affected, how can that be considered small?

Either way, this time we survived, and we survived because we learnt. Back in June and July we experienced these issues with EBS so we did something about it, now why didn’t everyone else?

How Alfresco Cloud Survived

So back in June and July we were heavily reliant on EBS just like everyone else, we had an EBS backed AMI that we then used puppet to build out the OS, this is pretty much what everyone does and this is why everyone was affected, back then we probably had 100 – 150 EBS volumes so the likely hood of one of them going funny was quite high, now we have about 18, and as soon as we can we will ditch those as well.

After being hit twice in relatively quick succession we realised we had a choice, be lazy or be crazy, we went for crazy and now it paid out. We could have been lazy and just said that Amazon had issues and it wasn’t that frequent and it is not likely to happen again, or we could be crazy and reduce all of our EBS usage as much as possible, we did that.

Over the last few months I’ve added a numer or posts about The Cloud, Amazon and Architecting for the cloud along with a few funky Abnormal puppet set ups and oddities in the middle. All of this was spawned from the EBS outages, we had to be crazy, Amazon tell us all the time don’t have state, don’t rely on anything other than failure use multiple AZ’s etc etc all of those big players that were affected would have been told that they should use multiple availability zones, but as I pointed out Here their AZ can’t be fully independent and yet again this outage proves it.

Now up until those outages we had done all of that, but we still trusted Amazon to remain operational, since July we have made a concerted effort to move our infrastructure to elements within Amazon that are more stable, hence the removal of EBS. We now only deploy instance backed EC2 nodes which means we have no ability to restart a server, but it also means that we can build them quickly and consistently.

We possibly took it to the extreme, our base AMI, now instance backed, consists of a single file that does a git checkout, once it has done that it simply builds its self to the point that chef and puppet can take over and run. The tools used to do this are many but needless to say many hundreds of of lines of bash, supported by Ruby, Java, Go and any number of other languages or tools.

We combined this with fully distributing puppet so it runs locally, in theory once a box is built it is there for the long run; we externalised all of the configuration so puppet was simpler and easier to maintain. Puppet, its config, the Base OS, the tools to manage and maintain the systems are all pulled from remote services including our DNS which automatically updates its self based on a set of tags.

Summary

So, how did we survive, we decided every box was not important, if some crazy person can not randomly delete a box or service and the system keeps working then we had failed. I can only imagine that the bigger companies with a lot more money and people and time looking at this are still treating Amazon more as a datacentre rather than a collection of web services that may or may not work. With the distributed puppet and config once our servers are built they run happily on a local copy of the data, no network, and that is important because AWS’s network is not always reliable and nor is their data access. If a box no longer works delete it, if an environment stops working rebuild it; if amazon has a glitch, keep working, simple.

Sysadmins in a Developers world

October 17, 2012 / / No Comments

It’s all back to front

Well it was about 9 months back when I was touching on Developers in a sysadmin world and my initial thoughts were along the lines of we are better at different tasks, and after spending a week doing only development I am of the same opinion still.

Over the last 6 months we have had our solitary developer, coding away making great things happen, predominately developing a portal that allows us to deploy environments in 15 mins vs the 2 days it took before and the whole things is very pretty, it even has its own Favicon.ico which we are all pleased about. In addition to just deploying, it also allows us to scale up and down the environments it creates and despite constant interruptions it is coming along really well and in the next month we will be providing it as a service to the engineering teams to self serve.

As more and more of our tools are developing we are also in-housing more and more of our tools. As the regular readers know I do dable with the odd slightly more complex program than the average sysadmin might tackle. When we are faced with a situation such as monitoring the operations, by this I mean, the number of user growth week on week and the cost of running the environment(s) it just made more sense to do it our selves. There are tools out there that provide various dashboards like Geckoboard which can all do approximately 80% of the job, but it’s that last 20% that adds the usefulness, as such we are trying to develop tolls that are pluggable and extensible and support multiple outputs. For example the Metrics report we have will also support Geckoboard, Graphite, Email and probably have it’s own web interface.

For us it is becoming more about having the flexibility to add and remove components and keeping the flexibility around it, this introduces challenges with what ever being written needing to be pluggable and easy to maintain, which often make sit complicated.

I used classes, as a necessity

Typically when I program there is not much need for classes or even objects for that matter, a simple var and some nice loops and conditional statements would be plenty. Well not so much anymore, The last project was metrics and as with other projects I got it working within a day or two, and I hated it, it took over 30 seconds for it to run and generate the report I needed but not in the right format and then the level of detail in the metrics was not high enough, it could manage weekly but it was not good enough.

I decided that I’d have a chat with a few developers to help with the structure of the application, at first I was dubious, but it turned out well. The key step which I wouldn’t have made until it was a real problem was to separate out the the tasks that gathers the raw data, the tasks that manipulates the data into useful numbers, the bit that stores the data, the bit that manipulates the data into useful numbers and then finally the bit that outputs the pretty data.

This was an evolutionary step, I would have got to the point of understanding the need to separate each step out but not until it had become a real big pain many months later. Another advantage of splitting it out was how much simpler each step was, there were classes defining methods for getting data that were being used in classes to format the data that were being used… you get the idea. Rather than being one class to connect to amazon, manipulate the data and return an object that could be used to generate the metrics everything was done on much smaller steps. As a result it was a lot easier to write small chunks of code “that just worked” and it made debugging a lot easier, and I feel like I progressed my understanding, and this is always a good thing.

Who should do what

I touched on this in my other post, but I want to amend it based on a better understanding. To summarise I pretty much said as it is, Developers develop, sysadmins admin. They do, and certainly that should be their focus, but I think there is a lot to be gained from both points of view when pushed to work in the others world.

Before our developer joined the focus was on making the build, test and release process better, after forcing the developer to do sysadmin work for a month or so while the team was trying to grow and cope with the loss of a team member, it became clear that the time wasted for us all was not getting a build though but by us not being able to paralise the testing or being agile enough to re-deploy an environment if it was not quite right. These steps and understandings would not have happened if we didn’t encroach on each others work and gain the understanding from the other persons perspective.

Summary

This is what DevOps is really about, forget sysadmins doing code, forget about developers doing sysadmin work, it is about us meeting in the middle and understanding the issues we each face and working together to solve bigger problems.

Release consistency

October 3, 2012 / / No Comments

It’s gotta be… perfect

This is an odd one that the world of devops sits in an awkward place on but is very vital to the operational success of a product. As an operations team we want to ensure the product is performing well, there’s no issues and there’s some control over the release process, some times this leads to long winded and bloaty procedures that are common in service providers where people stop thinking and start following, this is a good sign a sysadmin has died inside. From the more Development side we want to be quick and efficient and reuse tooling. Consistency is important, a known state of release is important, the process side of it should not exists as it should be automated with the minimal interaction.

So as Operations guys, we may have to make changes ad-hoc to ensure the service continues to work, as a result a retrospective change is made to ensure the config is good for long term; often this can stay untested as you’re not going to re-build the system from scratch to re-test, are you?

The development teams want it all; rapid, agile change, quick testing suites and an infallible release process with plenty of resilience and double checks and catches for all sorts of error codes etc etc, the important thing is that the code makes it out, but it has to be perfect, which leads on to QA.

The QA team want to test the features, the product, the environment, combinations of each and the impact of going from X to Y and tracking the performance of the environment in between. All QA want is an environment that never changes, with a code base that never changes so they can test everything in every combination, those bugs must be squashed.

Obviously, everyones right but with so many contradicting opinions it is easy to end up in a blame circle, which isn’t all that productive. The good news is we know what we want…

  • Quick release
  • Quick accurate testing
  • Known Product build
  • Known configuration
  • No ad-hoc changes to systems
  • Ability to make ad-hoc to systems
  • Infallible release process

All in all not that hard, there are two issue points here. Point one, infallible releases are a thing of wonder and can only ever be iterated over to make them perfect, in time it will get better. Day 1 will be crap, day 101 will be better, day 1000 better still. point two, you can’t have no ad-hoc changes and the ability to make ad-hoc changes, can you? Well you can.

If you love it, let it go

As a sysadmin, if there’s an issue on production I will in most cases fix it in production, if it is risky I will use our staging environment and test it on there, but this is usually no good as staging typically won’t show the issues production does, i.e. all of its servers would be working yet production is missing one. This means I have to make ad-hoc changes to production, this causes challenges for the QA team as now the test environments aren’t always the same, it then screws up the release process as we made a change in one place and didn’t manually add it in to all other environments.

So, What if we throw away, production, staging or any other environment with every release? This is typically a no-no for traditional operations, why would you throw away a working server? well it provides several useful functions:

  1. Removes ad-hoc changes
  2. Tests documentation / configuration management
  3. Enhances DR strategy and familiarisation with it
  4. Clean environment

The typical reason why the throw away approach isn’t done is due to a lack in confidence of the tools. Well, Bollocks. What good is it having configuration management and DR polices if you aren’t using it? if in an operational place now you are making changes to puppet and rolling them forward you achieve some of this, but it’s not good enough, you still have the risk of not having tested the configuration on a new system.

With every environment being built from scratch with every release, we can version control a release to a specific build number and specific git commit which is better for QA as it’s always a known state, if there’s any doubt delete and re-build.

The release process can now start being baked into the startup of a server / puppet run so the consistency is increasing and hopefully the infallibility of the release is improving, adding to this a set of System wide checks for basic health, a set of checks for unit testing the environment, a quick user level test before handing over for testing then it’s more likely, more often that the environments are at a known consistence.

All of this goodness starts with deleting it and building fresh, some of it will happen organically but by at least making sure all of the configuration / deployment is managed at startup you can re-build, and you have some DR. Writing some simple smoke tests and some basic automation is a starting point, from here at least you can build upon the basics to start making it fully bullet proof.

Summary

Be radical, delete something in production today and see if you survive, I know I will.

Open source architecture

September 26, 2012 / / No Comments

It all starts with an idea

When you sit down and start to think about it, we, as humans, are very good at learning new things and using our understanding to progress and do new things based on what we learnt. The founding principle is everything I know can be expressed thusly:

“What I know” = (“What I learnt” – “What I forgot about what I learnt”) + My experiences

So based on this every time I do something new I learn something, I then forget a proportion of that but have hopefully formed a subconscious opinion of it and have now gained an experience of that.

One of the ways I learn more is to read, use the internet to find solutions and adapt them to my current situation, sometimes that results in an experience or something I’ll actually retain and will re-use at another time.

Now when I’m often learning I am also working so I apply the “what I know” to the situation I am in with a dash of learning and experience and with some healthy collaboration end up at a solution, now isn’t that the key? collaboration is when we start sharing our “what I know” and our experiences of it to come to a solution that often removes the pain points that each member experienced in the past but potentially introduces more.

So that leads me on to the idea, what if we open source our architecture? Does that provide value? I think so, we have spent months years maybe on our architectures getting to a point that we as individuals are happy with them for various solutions, how cool would it have been if you could go to google and type “Open source architecture for hosting java based cloud solutions” and have it come back with something other than adverts for cloud solutions, maybe diagrams of the solution, maybe configurations, guidelines, bullet points directions of use etc etc. Wouldn’t that be like collaborating with others on your architecture, getting more experience and more knowledge to discuss the idea and suggest improvements. All of a sudden your one or two person IT team is now 30 or 40 people discussing the architecture, if just one of those people comes back with something that isn’t crap and is beneficial doesn’t that make it worth while?

Is there value in it?

I keep asking myself this. If every time I wanted to do a new “solution” there were “boxed” architectures for hosting applications in Amazon or setting up a data centre infrastructure I could pick it off the shelf, read some foot notes or supporting documentation understand the limitations and contraints of the solution and then adapt it to meet my specific needs.

I may even discover something that specifically could be contributed back to help out. For example, maybe Amazon EBS volumes become increasingly unstable so I fork the architecture and release an instance store version that then removes the dependancy on EBS or maybe a bug is found with ELB’s and access that causes a security hole, what ever it may be there a continuous feedback loop enabling everyone to benefit from the “what I learnt” and “My experience”

Okay, so in theory it seems okay, but, everyone has a unique situation so what ever is open sourced is potentially a waste? I don’t think so. Most of the principles behind what is done in any environment are the same, they are just tailored, I imagine most Corporate data centres have a number of distinct networks, probably a DMZ for web facing applications, maybe a more standard Application network for everything else that is not public facing or is internally accessible and lastly a secure area for DB servers or other important things that can never be accessed directly.

I think that’s pretty common, so the Open source architecture would depict this with maybe a dotted line to show A.N.Other network, maybe for Development, or for Testing but it’s a good starting point. I think the starting point coupled with the usage notes, diagrams design principles and documentation should be enough to get people up and running and with a decent architecture.

Summary

I’m sure there’s something in this, I know it will be beneficial, I worry that people will switch off and not take the architectures and adapt them to their specific needs, I hope in most cases that won’t be necessary, but there will always be fringe cases. Hopefully over the coming months this will turn into something more than words, but only time will tell.

Technology archeology

September 19, 2012 / / No Comments

What!?

We’ve all been on a archeological dig from time to time, faced with an environment that no one knows or an installation no one understands and all we’re left to do is dig our way through the evidence to uncover the truth; and much like real archeological digs we only ever come to a truth that we understand not the actual truth.

I don’t think there’s really anything that can be done to mitigate these situations, sure, you could make sure you document the system fully, write every little detail down and have a structured document a child could follow; no one will read it until after the catastrophe it could have saved happens; so you can’t document your self out of the problem.

Well there’s always hand overs and training, and for that to work you just need to get someone to have the same experiences and history as yourself and then two or three months of going over the same stuff and then there’s a small chance that they may remember some of that knowledge when they need to.

So you can’t write it down, you can’t hand it over and you can’t be around to deal with the problem yourself, so what can you do?
Not much, it depends on how much time you have, but the only real way to hand things over is when they fail let someone else deal with it.

Planning for digs

So how can this be mitigated, well I believe standardisation, problem solving and starting over are the best things to do. Lets look at them in a little more detail.

Standards

Everyone has standards, some more than others, what ever the standard is, stick to it, whether you agree or not. By having a standard way of implementing cron jobs, i.e. Always putting them in /etc/cron.d/ rather than crontab or a mixture of cron.d and cron.daily, putting all of your shell scripts into a common directory. Always writing scripts in a certain language, even if it means it will take longer and you have to learn something new…

This doesn’t sound like much, but it means everyone knows where to look to start dealing with problems, there’s no special hidden away super squirrel scripts somewhere that people won’t find.

Problem Solving

Quite simply, Get better. Being able to problem solve quickly is a difficult skill to do accurately but practice makes perfect and the more you do it the easier it gets. This is useful for dealing with problems that are unexpected and the best way to get better is to make sure the person that set it up isn’t the person debugging it all the time, share the workload around and let people be in the deep end struggling while you’re around to help rather than struggling on their own.

Starting over

The biggest barrier to “adopting” a legacy system and owning it for problem analysis is that the people that are trying to support it no longer care, they didn’t put it in, it was put in by a bunch of cowboys badly, with no documentation or poorly written docs and no hand over! It doesn’t matter how good the docs are, you can’t document everything so you always assume a basic understanding, and you can’t tell people that don’t want to listen. So the easiest way to get new people to want to look after systems they didn’t set up or had no part in is to let them do it their way.

So, during the first few months introduce them to the system explain it all and accept the little chirps about there being a better way; at this point they should have a good understanding of the current system and maybe the chirps will disappear, maybe they won’t. If there’s still a lack of adoption it may be worth pretending there are problems with the system and that it’s worth looking at a “better” way of doing it, as such, let them lead the technology charge and just make sure that the system provides the same functionality as before, now all you need to do is learn their system!

Summary

I’m not saying don’t document… I am saying don’t spend a long time on it, bullet points, basic pointers and directions, backup / restore enough that if someone skim reads it looking for information it at least takes them to the next step. I’m also not saying don’t do handovers, do them! just accept that it’s in oen ear and out the other, but at least you tried.
Bear in mind that people that are new to the group have big ideas (like you did) and want the best solution, the thing they’re missing normally is the history, “why was it done that way?”, just remember that they need the history so they can understand why something may be bad and why it was not made better at the time.

Hopefully these little things will help the technological archeological dig not be as deep or take as long.

Not putting in the time and effort

July 20, 2012 / / No Comments

Last week…

I realised I basically was running out of ways to say “always put in extra time and effort when you can, it will reward you”, now if that is too concise you can read the full blog here. As a result I thought I’d drag it out to a whole new post about not putting in the time and effort, after all your time is valuable.

How much time do you have?

Really, how much? I use to think I had lots of time, then I realised that I don’t have any time really, the only thing that changed was I got in a relationship and it was more important to be doing other things than sitting at work late into the night or watching tv all the time.

I then started to realise that something had to give, I could keep working or I could retain a relationship, I chose the relationship. I then realised that I actually had more time than I originally thought, I was just wasting hours each week, I like watching tv like the rest of us, but I did this amazing thing that I believe Women take for granted, I started to multi task, only ever giving half of my attention anything, but quite frankly, it’s tv…

So, again, how much time do you have? probably more than you realise.

where to spend your time and effort

So as hopefully we all came to the same conclusion, you can probably find time to do some things, certainly you can find time to do important things. As with last week I was touching on using your time smartly then but to invest the time and effort into improving your career, well what happens if you chose to take that drive and energy and apply it internally? So rather than just looking at your career, look at making time to spend on doing things you want to.

This year I decided that I would do a few things, one, maintain a blog, which I hadn’t done for a long time and to make things worse I thought writing two articles a week was a good idea, based purely on the fact that someones articles I read on the register writes two a week. So In addition to the blogging I took on number two, create an open source project, I had always wanted to do one / contribute to one and I didn’t ever want to ruin a proper project so just did my own.

If there’s something you’ve always wanted to do then why don’t you do it ? You don’t have to slave away at work if there’s no benefit to it, especially if you could spend that time doing something else. I’m not saying these projects will progress rapidly but you will be able to progress them now and then. What more do you want?

So other than being distracted…

The projects can still benefit you career wise, if nothing else it’s a talking point to discuss with people and show that you do put in the time and effort into projects, work related or not.

Let us not forget that a lot of businesses start off as a project for fun and enjoyment and not purely a business venture, Google was a research project called BackRub Facebook was pretty much created to prove a point that it could be done quicker and simpler by Mark than allowing harvard to take 2 or 3 years over it ( Here ). I could go on, but in short the best companies around are created out of a need / desire to do something different and outside of the norm, all squeezed into those little chunks of time that you currently don’t have.

Summary

You have the time, you just need to put the effort in.

Putting in time and effort

July 13, 2012 / / 1 Comment

Every now and then…

A decision needs to be made, do you value your time or your effort? I know when I first started out in work I had little choice but to put in the hours. At some point we all have to just put in the hours for one reason or another. For me putting in the hours was about working 30 hours a week and doing a full time college / university course and still finding time to educate myself out side of college or university. During those days I had time on my hands I didn’t have a CV full of skills or jobs that made me look attractive to employers so I had to put in hours to make myself achieve what I wanted to and to ensure I could continue achieving more.

Occasionally there will be those moments, rare moments where you look at something that is working, no real issues with it and you realise it’s not “right”; you identify some issue or room for improvement and you give it some thought come up with a few ideas and then just implement the best one. This is putting in some effort, there was nothing really wrong but you decided it needed to be better, it needed to be improved some how and then it was done.

When I talk about effort I am not talking about slogging away for hours at a time I’m talking about people thinking about improving a situation on their own. It is incredibly easy to just sit there and solve problems that are presented, it’s something different to try to solve problems that no one is screaming about or worrying about but provide huge benefits to your daily work.

When to use each

Let’s start by clearing up something, you as an individual work for a company to earn money, the company works for the share holders to get them money. At no point is the company working for you, sure companies will pay you a salary and throw bonuses at you if you successfully navigate some hoops and give you a raft of benefits on the side. Now going back to earlier, at which point was any of this putting in extra time or effort to make you happy? give you better experiences or to increase your value.

Bear that in mind, I will probably contradict it in a moment.

In short, you care about you and you need to make a call whether investing the extra time into a work related task is worth while or that putting in some effort is worth while, no one else can do that, and anyone that tries should be shot down vigorously.

If you have some spare time where maybe your wife is out or you were just going to sit down and watch tv and you decide that on this day trading a couple of hours of home time to for work time would provide some benefit to you then you should do it. It’s not about putting in 12 hours days, it’s about using the time you have better, and when needs must and you want to, never feel pressured.

As for effort, you should be doing this all the time, yes you can have a easy time do the job and move on and scrape by doing the bear minimal, and sometimes this is a necessity due to time constraints, so don’t feel bad about it for starters. Normally to go that little bit further and add extra functionality to something is only a small amount of actual effort so it is worth identifying some “quick wins” in terms of effort, it may take away a bit of that extra time you put in but it is always valuable.

Now, what about if you work in an environment where they are rewarding you in ways that are “non-standard” ? Maybe after a hard month of effort they let you expense a meal for you and your wife (I’ve had this happen twice and it’s really nice mainly for my wife who is a work widow) or maybe they decide to fly you and everyone in the company to a Mediterranean island and let you do a treasure hunt across an island on a jeep (this has also happened). Well, I’d say firstly, if you’re being rewarded for the time and effort you put in then grab it and go with it, take what you can from the experience you never know where it will take you, but for sure, set targets for your self and only rarely re-address them and only if the company is working for you.

So if you are that lucky and even if you are not, choose to invest time and effort into the things you do if for no other reason than so you can say you did it.

When is enough, enough?

July 6, 2012 / / No Comments

Every now and then…

There’s a time, once in a while when someone get’s on your nervers, if you’ve met me, I’m probably that guy ;-)
Well this is called life, and there’s not much point getting worked up about annoyances, or people that get on your nerves, unless you have spare time to waste on such activities anyway, personally I’d rather just move on and wait for sunny times. That isn’t always possible, either due to your own state of mind or for some reason the ‘annoyance’ (yes I just referred to a person as an annoyance not granting them any dignity or space or time or energy ) won’t give up and move on, again, this is called life.

Is it a long term or short term affect

From my experience I would say that there tends to be two types of ‘annoyance’ either it is short term or long term; it is quite important to identify this as it means you can take appropriate action. For example, if there is someone in your team that is traditionally a happy go lucky sort of person and one day they seem to just be an arse, sorry ‘annoyance’; then you’d probably be right to let it slide and do nothing about it, this is very low impact for you effort wise. If there is a member of your team that seems to take constant glee in pushing your buttons or pointing out your flaws, creeping behind your back and so on, this is slightly more longer term.

Short term ‘annoyance’

So with those that you would classify now as short term annoyances there’s always multiple options depending on how much you like the person. If you’re not too keen on the person or you don’t have to spend much time interacting with them you can just brush off the odd comment, if it is someone you have to work with closely you could simple ignore it and wait for an opportune moment to ask if everything is okay. Be warned, if you ask someone “is everything okay?” and you do it disingenuously this will fall back and hurt you, if you dont really care don’t ask, you’ll make it worse.

Long term ‘annoyance’

I hate these, mainly because it takes a lot of effort, either positive or negative depending on the brutality of the ‘annoyance’

If there’s someone in your team you just don’t see eye to eye with it’s a struggle, you’ve had a few handbags, disagreements etc already and you both pretty much agree you hate each other. By this point it doesn’t matter if the other person is being nice, in your eyes it’s two faced. There’s only really two ways to deal with these types of people, either be really nice, or be really devious in both cases be meticulous, and because I don’t want to let people in on how to “eliminate” these problems we’ll focus on the nice tact, it also drives them crazy!

I’ve been in a position where there’s been someone I really couldn’t get on with, At first it was handbags, but then I started to ask myself what causes someone to be that obnoxious. I started to look at their history, who they had worked for, where they had come from, how they had gotten to where they are, what structure their personal life had, how they dealt with pressure, what influences their decisions, what is important to them, how do they communicate, what are their tells and so on, the list is endless. I believe if you have a long term ‘annoyance’ you have to know them inside and out, this is great for the being nice approach, you have no choice but to get to know them as an individual. It will take weeks of being nice and ignoring snipes and odd remarks but you should see a genuine improvement in the relationship, once that happens you will find that it no longer bothers you what they do you can just be nice and say things like “I noticed you forgot this, so I did it for you” or “how was your weekend” see this is caring and sharing…

Summary

As much as you may want to change someone else you can’t, the only thing you have any control over is yourself so you can make yourself be nice, you can make yourself ignore comments and eventually things will get better. On a side note while being all nice and kind and not being mean it certainly wouldn’t hurt to have a good record of what is said in meetings, who committed to what and if you have a ticketing system, use it. Hopefully you’ll never need it, but if you do you have it, and because you wrote it down and the other person saw you doing it they have even less to go on.

So when is enough, enough? at the point you decide to start changing yourself and not others.

Loving what you do

June 22, 2012 / / 2 Comments

A while back…

I touched on that you should love what you do Here. I also found myself over the last few months wondering if I was still loving what I do, the short answer is no, the longer answer is yes. As those of you that read the other post will know, you should always be doing what you love else what’s the point. I identified a long while back, maybe 6-9 months that there are certain areas of my role that I prefer over others and a few years ago there’s certain things I really enjoy doing. So before going to much further I do enjoy my work, it is challenging and rewarding and that should be enough, but there was a little niggle that was telling me that something wasn’t right with what I was doing.

I had a sit down and a think and I realised I enjoy doing the technical hands on elements to a point, after that point the enjoyment drops off rapidly. For me that point is when the system starts to bed in and becomes more routine, it’s not an element I enjoy of the role but it is something that needs to be done, it is very important in fact. I’ve been really lucky that I don’t seem to be around long enough with any projects to see them through to completion, or when they are completed I’m then working on another large project and tend to skip the day to day work around that, so although there are elements I find dull, I don’t mind as they don’t normally affect me for too long.

There is also something else I know I like doing and I’d like to do more of. At a previous employer I was given the opportunity to do some mentoring of a graduate, and I found I really enjoyed that, I found it much more rewarding than the day job, I felt like I was actually helping someone achieve more, which is very rewarding.

Bring what you love to work

With all of this in mind I started to think about what else I could do within my current job role to give me that feeling again. Well the obvious answer for me was to get some graduates to help out and give me some people to mentor which I enjoy doing. Unfortunately this time that plan did not work out, but it is getting closer and one day will work out.

So this time that plan didn’t work out, so I’m now in the position of finding another way to make that happen, maybe graduates was a bridge to far. Maybe there is something else that could be done to give the same feedback, I will of course need to think about it a bit more.

The main point is that if there is a small element missing from your job, in my case mentoring, find a way to bring it into your job and don’t get disheartened if one method doesn’t work. Keep trying. It is important to keep trying especially if there is only one element that makes it not quite the right role for you.

Don’t run away

Other than trying to bring the elements you do want into your role you could try and change role, this isn’t a bad idea but think about it. I recently started considering other roles that had more of a leadership part to it that would allow me to mentor people and help develop them but as with most people the roles I was looking for didn’t exist in my current company.

The challenge than changes from trying to bring one element into a all round good role to ensuring that all elements of a potential new role meet or exceed your current position. For me the exercise was relatively pointless as all the other elements of, my role are perfect for what I want to do, but if you are not in the same position then moving on may be the right thing for you, but don’t do it to try and make one element better as more than likely the other good areas of the role will probably suffer, it’s all about swings and roundabouts.

Summary

Don’t try to change your job if every element is good about your current role except one element, instead try and find ways to fix that element in your current role instead. There is always a way.

Stress

June 15, 2012 / / No Comments

We’ve all been there at some point

Stress isn’t fun, there’s a point before the un-fun stage of stress that seems to help make us a little more productive. Everyone needs a mechanism for dealing with it else they will just get overwhelmed at some point and then enters the not so fun side of stress.

It was a few years back when I had my first stress related incident, working on a project trying to get a delivery done by a certain time, working some 15 hour days to help get it done. What did it take for all the stress to build up and make me break as a person? Just one project manager being an arse, insisting that more is done and we should work longer hours; he was so out of touch with the amount of effort that was already going in. I had to walk out of that meeting I was in tears within 30 paces, I punched one of those cubical divide things as I was walking past (the cap on the top flew off and hit the ceiling which drew a little more attention than needed!) and I ended in the toilets sobbing like a child. Welcome to stress.

Identifying stress

I had no idea I was getting stressed, it was sort of like a ninja attack one minuet fine, the next blubbering wreck. HowStuffWorks is pretty good for this. In short, everything stresses us out, everything. Not being able to go to toilet, not being able to meet up with a friend, having coffee, not having coffee and so on. So identifying what causes stress is quite straight forward… Everything. As the article say’s, you can’t always remove the stress but you can reduce them.

After my “episode” I spent some time thinking about what really tipped the balance for me, I looked at what was going on at home and work, what was different about the work I was doing now to 6 months ago and so on. For me it seems what triggers the most stress is not being able to do what people expect of me, the normal day to day work is fine and project work is fine. Probably worth mentioning that before my “episode” there was two of us on the project doing the same sort of things, unfortunately my colleague had just been signed off with stress as well, so my work load doubled and it was already pretty high. This was another contributing factor, not because I had to pick up my colleagues work but because my colleague wasn’t there to share the mumbling about the project which helps reduce stress.

So if you find your self stressing try and identified what’s changed, it may not be one thing it could be a combination of things causing it, but as with most things until you know what is causing it you can’t do much to fix it.

Dealing with stress

I use to deal with stress by putting a very clear boundary between work and home, I go into work typically between 7:30 and 8am and leave normally around 6pm with the occasional desertion from work from 3pm onwards if I can. when things get too busy and I start noticing I’m beginning to get stressed I tend to start brining the hours in, I start making sure I do my contractual hours more so than the hours I typically do. Likewise I don’t take work home with me, I very rarely do work in the evening or at weekends unless it’s something I want to do like a special project I kicked off.

So the clear boundaries help separate the stressy part of the day from the non stressy, reducing the stress caused by work.

The other, and the most important thing is to stop caring about the work, yes it’s important, yes it needs to be done, but it’s not all on you, you are not the only person involved in this, it’s a team game. This helps reduce the worrying element of trying to get it out and takes some of the self imposed pressure off of you, you do need to care about the quality of the work you do, but not so much what that work is.

The other most useful step is asking for help, I use to be very bad at asking for help; even at school I was being told to ask for more help and the same goes for work. Last time I had a look I wasn’t wearing my underpants on top of my trousers, or had a big ‘S’ on my chest within a shield, or was a fictional comic book hero, I’m assuming you aren’t either. By asking for help you may get some, you may not; but you stand a better chance than not asking at all.