Interesting Machine Learning Resources
Generative Adversarial Networks
Also known as GAN or DCGAN (Deep Convolutional GAN).
Short explanation of GAN, by Yan LeCunn pdf.
Detailed article about Generative Models / GAN by OpenAI, pdf.
Racing Car simulator
With Genetic Algorithms and Neural Nets
Finished Machine Learning Course
I had couple of ML-related courses at University and used it in practice, but, still it was very interesting to review and refresh the core concepts.
Especially from such a badass ML expert as Andrew Ng, with lots of practical examples and advices.
My Notes and the Course by Andrew Ng.
- Supervised / Unsupervised Learning.
- Linear Regression, Logistic Regression, SVM.
- Neural Networks.
- K-means, PCA, Anomaly Detection.
- Recommending Systems, Collaborative Filtering.
- Bias / Variance, Regularization, Precision / Recall, Learning Curve etc.
- How to gather, amplify and generate Learning Data.
Gold Coast
Interesting algorithms
Manly
Party
Nice visualizations
Health
Bondi Beach
Sydney
Project Health
An interesting exercise that tries to reduce risk of the failure and identify problems early.
The Project Team has been asked to answer couple of questions about the state of the Project. To make it more fair the Team has been split in two groups and each of groups has been asked independently, without knowing the answers of other group.
The result was quite unexpected, the answers where different for almost half of the questions. You can see it on the picture, same questions answered with different colors.
Food supplements
My christmas present, should be enough for couple of months.
Below are list of most usefull food supplements that have proven benefits (I'll add links to exams and proofs later).
Core supplements
- Multivitamins, Microelements - general health, any good multivitamins, the more vitamins and elements the better, also make sure there's the Iodine.
Optional supplements
Glucosamine Chondroitin MSM - joints support.
Liver detox - herbs extracts for better digestion.
Grape seeds - immune support, antioxidants.
Coenzyme Q10 - cardiovascular system.
Trip to Europe
Had business trip to Europe for 2 weeks, visited Barcelona, Berlin and Amsterdam.
Barcelona
Berlin
Acceptance Testing for Web App
I'd like share an approach we use at Qubell to ensure quality of our product. A set of practices that allows us to use Acceptance Testing in simple and efficient manner.
Note: this article is my personal opinion and can't be associated with Qubell's official view on the matter in any way.
Test Case
Let's try to express the test case with plain english first. A little about the product itself
- the Qubell Platform simplifies deployment. You write requirements for your Application, hit the launch button - and your Application get delivered into production.
On the screenshot below you can see list of sample applications.
Making Crawler Fast and Reliable
So, as I told in the previous article - the basic version of the Crawler worked well and proved to be usable. The problem - it was slow and unstable.
To make it fast we need to run it on multiple machines (about 5 - 20). And to make it stable we need to figure out how to make reliable system from unreliable components.
The unreliable component - Crawler uses Browser Emulator (Selenium) with enabled JavaScript to properly render content of sites. And it consumes lots of resources and frequently crash and hangs (it is not very stable by itself and what's worse - there maybe invalid HTML or JS on different sites that can crush or hang it).
Going distributed
Multiple machines instead of just one make things a bit complex because couple issues arise:
- Provisioning and deployment couple of tens of machines. I don't want to do it by hands.
- Handle crashes of machines and heal after it. Crawler should be robust and continue to work if one of its nodes crashes, and pick it up again when this node get fixed or new node get added.
- Detect if one of its nodes hanged and need to be rebooted.
Sochi Svetlana
Crawling JavaScript Sites
Recently finished project - Web Crawler for JavaScript sites. It crawl eCommerce sites and collects information about the products. You can see parsed products for one of such eCommerce sites below.
Usually crawlers browse site pages, collect HTML from it, parse and extract some data.
But, in our case we can't use such approach - because sites we are interested in aren't usual sites. There are no any useful data in HTML sent by server, because they rendered using JavaScript in the Browser. And to parse such sites you need full Browser with working JavaScript Engine.
Browser Emulator
Sochi Ahun
Sochi Zapolyarye
MongoDB Performance Tuning
Best feature of MongoDB is not it's performance but simple and flexible data model. So, let's say you build prototype - you concentrate on the big picture - the product itself and ignore little things like performance and db indexes.
Later you deploy your product into the wild users came and it starting to get slow. You need to add indexes, to do so you need to know data usage patterns. Doing it manually by searching codebase is boring and not very productive. Thankfully MongoDB has Profiler - all you need is to enable it and it will give you all details about slow queries and what indexes you need to add.
I like this approach very much, because it fits iterative & lean development very well - you always concentrate on the most important things at the moment. At the first step most important thing is to experiment with the product and features without being distracted by performance issues. And flexible data model of MongoDB comes very handy to that. Later you deploy product into production and can use its Profiler to zoom to more fine grained performance details.
Example
Ruby on Rails vs Node.js
I wanted to see benchmark that more or less close to real life, not just measuring how fast it can stream data via http.
So I created application that query some text from remote HTTP service (service delays each request for 200ms) and render HTML page using that text.
It simulates how Web Framework performs when it needs to wait for response from DB or other services, how fast its templating engine is and also how fast it is itself.
And hit it with wrk -t2 -c100 -d10s http://localhost:3000 you can see results
on the picture.
3782 vs. 2914 hits, average response time is also very close.
Cart Widget
Created simple Cart Widget http://salejs.com that can be included into any HTML page and turn it into Shop.
There's also tutorial and screencast explaining development step by step http://jslang.info/projects/salejs
California
Prototyping Mobile Web Application
We decided to spend one evening prototyping very simple and small but unusual thing for our product.
I choose to build very basic mobile prototype. Actually our app is already responsive and can work on mobile devices, but it doesn't looks very good. I wanted it looks like a native application.
It took me about 4 hours to build this prototype (Backbone, Ratchet and CoffeeScript), you can see it live here code on GitHub.
