The parameter value is a parameter not a variable inside the function so it's the function call that creates it. The value itself isn't really decremented. Instead a new parameter is created that has is equal to value - 1 and that's passed to the next call in the chain.
For example, if your code calls Factorial(5), then that code evaluates 5 * Factorial(4). The call to Factorial(4) evaluates 4 * Factorial(3), and so on.
Eventually you get to Factorial(1). At that point, the code returns 1. You can see the input value 1 in your WriteLine statements. (Good use of those, by the way.) You don't see 1 as an interim value because when value is 1, the code doesn't get to that WriteLine statement.
The end result is:
5 * Factorial(4) =
5 * 4 * Factorial(3) =
5 * 4 * 3 * Factorial(2) =
5 * 4 * 3 * 2 * Factorial(1) =
5 * 4 * 3 * 2 * 1 = 120
I hope that helps. Recursion is very unnatural and can be very confusing. And this is about the simplest example possible with one function calling itself directly. Other recursive programs use multiple functions that call each other in complex ways or that call themselves multiple times. Then things can be really confusing.
For some interesting examples, see:
Draw a fractal Hilbert curve in VB
This one is multiply recursive--a routine calls itself multiple times.
Draw a Sierpinski fractal curve in C#
This one is multiply indirectly recursive--four routines call each other multiple times. (I need to make a VB
version of this one. I thought I had one.)