Weekly Journal: Week 6, 2019

Well, the Australian year has officially begun (nothing really starts until after Australia Day!) – and knowing that I will need to hand over my intense code duties soon, I spent the week working on some more complex pieces of code.

More specifically, last week I spent a lot of time working on a solution for creating alternate product recommendations using SpaCy. The idea I had was to measure the Word2Vec distance between product attributes for each attribute a product would have against all attributes of all other products. The problem was though, that this solution is ‘brute force’ and turns out to be in quadratic time without using SpaCy – once I use Word2Vec, it shifts to polynomial time. I need to rework the solution to use a different kind of approach to improve the performance and handle scale. I will tell you what I came up with next week.

At the same time, our Technical Director started playing with my pipeline code – over the weekend he figured out a way to rewrite the entire pipeline so it could be ‘chunked’ and distributed. He simplified and streamlined some of the algorithms and basically threw out a bunch of my code. I felt bad for a few seconds – kind of shows who the better engineer is, right? However, once my ego had its little pity-party, I realised this was awesome – we now have a massively scalable data processing pipeline! I’m really excited to get started on V2 of this platform.

Below are some of the videos I was watching this week…

I love this video so much… Just shows how we are so easily caught up in hype of new concepts that are actually founded in old concepts. This video will blow you away once you see how old some of the ‘new ideas’ really are.
Next time anyone says ‘we should be science based and there is no place for spirituality’ get them to watch this video – not because it has anything to do with spirituality – rather, because it shows you how much we know versus how little we actually know why.
Because this guy is just amazing – and it’s beautiful.

It’s been a very busy week last week so I did not have as much time to read as I would have liked, so I am still working through the “Learning Computer Architecture with the Raspberry Pi” – which is just such an excellent book.

What else was last week? Well – actually spent some time on less glamorous tasks such as:

  • financial modelling to demonstrate ROI of a piece of data science work we were doing,
  • high-level project design and scoping
  • working on disaster recovery and business continuity planning (which is dry – yet essential!)
  • starting to get back into sales planning – this week I am on that in earnest!
  • starting to do some deployment planning for new orchestrated container services we are deploying in a few months. (ok, this was actually fun!)

Last week was also very full of Karate training – I really enjoy it – and last week was no exception (other than breaking a toe). Learning the art of Kumite (sparring an opponent) has so many analogs to other areas of life – one of the things I was learning was about classification of the opponent: if you make some broad behavioural categories there are some techniques you can use to identify which category an opponent would ‘fit into’ and if you learn strategies beforehand for dealing with these classes of opponent, you have a vocabulary you can draw upon in an instant.

Think about this, I can use a very simple gesture that does not commit me to an outcome, to better understand my opponent and then reconfigure my attack and defence strategy in a fraction of a second – based on (pretty much) a twitch. This is using categories and response routines to optimise our OODA loop by using simple signals to ‘orient’ and muscle-memory to ‘act’. In essence, the more this is done the more this becomes subconscious.

The key is using the edge of consciousness to introduce variability at speed into our response (and with the detection mechanisms we use) to throw the pattern matching in our opponent at the same time.

That’s about it for my retrospective… have a great week!

Regards
Joe

thinking.studio

Level 25, 88 Phillip Street
Sydney
NSW 2000
Australia

+61 2 8024 5975
[email protected]

©2003-2019 Pixolut Pty Ltd trading as Thinking.Studio
All Rights Reserved