Home

zhangyiqun

Thoughts, stories and ideas.

翻译 2014年以前 关于

19 Aug 2020
Knowing when to stop / 优化细节时最好知道何时结束

不断的提高时间ROI是件值得长期优化的事情,今天看到一篇文章作者讲的是在训练AI模型时怎么优化时间产出,虽然是机器学习领域但我觉得道理其实通用,因此把重点写了笔记分享。

在做分析时知道什么时候停止是件有点棘手的事情。这件事通常没有明确的结束点,你总是能在自己的模型中找到可以优化的参数。

In predictive analytics, it can be a tricky thing to know when to stop.Unlike many of life’s activities, there’s no definitive finishing line, after which you can say “tick, I’m done”. The possibility always remains that a little more work can yield an improvement to your model. With so many variables to tweak, it’s easy to end up obsessing over tenths of a percentage point, pouring huge amounts of effort into the details before looking up and wondering “Where did the time go?”.

Strategies

设置截止时间避免帕金森定律。帕金森定律是讲一件事情可以10小时做完也可以1小时做完。如果分配了10小时时间到只需要1个小时的工作上,最终也会花掉10个小时完成本来可以在1个小时就完成的工作。只要还有时间,工作就会不断扩展,直到用完所有的时间。

提前设定好可以接受的工作目标,达到预定的目标后便停止。这个办法实践不是那么好用,不确定性强的工作在开始前很容易就把目标定的过高或者过低。

收益递减到自己不能接受时停止。每次优化都能或多或少带来现实中的收益,如果收益和投入的时间价值不匹配则停止。我额外提一句,这个办法只是对特定工作适用,实际生活中有很多事情没法立竿见影,如果错误使用会使自己陷入三天打鱼两天晒网的处境。

Some strategies that can help you decide when to wrap things up might be:

  • Set a deadline -Parkinson’s law states that “work expands so as to fill the time available for its completion”. Having an open ended time-frame invites you to procrastinate by spending time on things that ultimately don’t provide much value to the end result. Setting yourself a deadline is a good way of keeping costs low and predictable by forcing you to prioritise effectively. The down-side is of course that if you set your deadline too aggressively, you may deliver a model that is of poor quality.
  • Acceptable error rate -You could decide beforehand on an acceptable error rate and stop once you reach it. For example, a self-driving car might try to identify cyclists with a 99.99% level of accuracy. The difficulty of this approach is that before you start experimenting, it’s very hard to set expectations as to how accurate your model could be. Your desired accuracy rate might be impossible, given the level of irreducible error. On the other hand, you might stop prematurely whilst there is still room to easily improve your model.
  • Value gradient method -By plotting the real-world cost of error in your model, vs the effort required to enhance it, you gain an understanding of what the return on investment is for each incremental improvement. This allows you to keep developing your model, only stopping when the predicted value of additional tuning fall below the value of your time.

from: Towards data science

翻译 2014年以前 关于