{"id":85,"date":"2020-09-04T02:38:59","date_gmt":"2020-09-04T02:38:59","guid":{"rendered":"https:\/\/blog.embeddedexpert.io\/?p=85"},"modified":"2020-09-04T02:39:38","modified_gmt":"2020-09-04T02:39:38","slug":"basic-derivatives","status":"publish","type":"post","link":"https:\/\/blog.embeddedexpert.io\/?p=85","title":{"rendered":"Basic Derivatives"},"content":{"rendered":"\n

In this lesson we are going to give a short revision of calculus. This will help to be able to picture backpropagation and gradient descent more easily in future lesson. You do not need to understand calculus to work with deep learning major of people using deep learning apart from the hardcore researchers of course do not understand the calculus aspect of backpropagation and gradient descent<\/p>\n\n\n\n

\"\"
The function plotted here says f of is equal 3a.
<\/figcaption><\/figure>\n\n\n\n

This means we get the result of the function f(a) by multiplying 3 by whatever the value of a is. In this graph the x axis represents the \u201ca\u201d values and the y-axis represents the f of a values. For example, lets say a equals 2, then f of a is going to be 6 since we multiply 2 by 3 to get f of a. Lets say we increase a by a tiny amount say 0.001 such that a becomes 2.001. <\/p>\n\n\n\n

When we compute f(a) again we shall get 6.003. We can realize that whatever a is f of a is 3 times that value. We we plot our two \u201ca\u201d values and their corresponding f of a values this is what we get. We can compute the slope of f of a by dividing the vertical change by the horizontal change. Over here we have named the vertical change as height and the horizontal change as width. When we perform this division we arrive at the answer 3. This is known as the derive. The slope and the derivative mean the same thing. This is often written as \u201cd\u201d \u201cf of a \u201d over \u201cd a \u201d and the answer is 3. Also I should point out that you don\u2019t always need to plot a graph in order to compute the functions. We shall define basic rules for finding derivatives later on in this lesson. Lets see another example.\u00a0 Over here we have the function y equals x squared. <\/p>\n\n\n\n

We computed the derivative by delta y divided by delta x.<\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n

Now lets take a look at some rules of derivatives.<\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n

The power rule states that to find the derivate of a function x raised to power n , the answer becomes n times x raised to the power n-1. If n = zero then the derivative is equal to zero because the derivative of any constant is equal to zero. If we want to find the derivative of a constant A multiplied by a function f of x. We can find the derivative of the function f of x and then multiply the answer by the constant. <\/p>\n\n\n\n

The derivative of the function f of x is sometimes written as f prime of x.<\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n

The derivative of the sum of two functions and f of x and f of g is equal to the derivative of f of x plus the derivative of g of x as we can see shown in these two examples.<\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n

We already mentioned the derive of a constant equals zero. The derive of a function multiplied by a constant is equal to the derive of the of the function only\u00a0 multiplied by that constant. We also said the derivative of the sum of two functions f of x plus g of x is equal to the derivative of f of x plus the derivative of g of x. <\/p>\n\n\n\n

The difference rule works the same way as the sum rule.<\/p>\n\n\n\n

The product rule says the derivative of the product of two functions is equal to the derivative of the first function multiplied by the second function + the first function multiplied by the derivative of the second function.<\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n

The derivative of sinx is equal to cosx. The derivative cos x is equal to minus sine x. The derivative of euler\u2019s constant e raised to the power x is the same e raised to the power x.<\/p>\n\n\n\n

The chain rule is the one that is heavily applied in back propagation.<\/p>\n\n\n\n

It states that we apply the chain rule by multiplying the derivative of the outside function by the inside function. <\/p>\n\n\n\n

It is often expressed as dz dx equals dz dy multiplied by dy dx.<\/p>\n\n\n\n

Or f of g prime equals f prime of g multiplied by g prime.<\/p>\n\n\n\n

Lets see an example.<\/p>\n\n\n\n

Lets say we want to find the derivative of the function bracket open 3x +2x2\u00a0<\/sup>bracket closed, all squared.<\/p>\n\n\n\n

We first find the derivative of the entire function which is 2 bracket open3x +2x2\u00a0\u00a0<\/sup>bracket closed and then multiply it by the derivative of the content in the bracket.<\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n
\"\"<\/figure>\n","protected":false},"excerpt":{"rendered":"

In this lesson we are going to give a short revision of calculus. This will help to be able to picture backpropagation and gradient descent more easily in future lesson. You do not need to understand calculus to work with deep learning major of people using deep learning apart from the hardcore researchers of course […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[15],"tags":[14],"class_list":["post-85","post","type-post","status-publish","format-standard","hentry","category-deep-learning","tag-deep-learning"],"_links":{"self":[{"href":"https:\/\/blog.embeddedexpert.io\/index.php?rest_route=\/wp\/v2\/posts\/85"}],"collection":[{"href":"https:\/\/blog.embeddedexpert.io\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.embeddedexpert.io\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.embeddedexpert.io\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.embeddedexpert.io\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=85"}],"version-history":[{"count":1,"href":"https:\/\/blog.embeddedexpert.io\/index.php?rest_route=\/wp\/v2\/posts\/85\/revisions"}],"predecessor-version":[{"id":86,"href":"https:\/\/blog.embeddedexpert.io\/index.php?rest_route=\/wp\/v2\/posts\/85\/revisions\/86"}],"wp:attachment":[{"href":"https:\/\/blog.embeddedexpert.io\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=85"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.embeddedexpert.io\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=85"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.embeddedexpert.io\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=85"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}