When I am writing I am not sure how deep into the story I need to be before I totally know what it is. There are those stories that I write where a fair chunk of it is unpacking the significance of the first chapter that I write. Others start vague and travel towards a knot of complexity, and then that knot needs to be untied before I reach the story’s end.
It is often suggested by the character and their logical growth, and the logical growth of the surrounding cast of characters and where the stories intersect – this is probably basic story writing 101, but I know some people plan all this out, and I am much more random and go with the flow. Sometimes an idea demands to be explored, but then I push it through the filter of a character’s perceptions or through the decision tree they would employ, and the idea changes shape.
I find that if plot doesn’t survive interaction with the characters then it isn’t strong enough, and it was merely the notion of it that was needed as a catalyst to push the story forward.
I navigate impasses by writing them into the story as problems for the characters to work out, and often they have a better idea of how to handle it than me. I listen to jazz musicians talk, or any good musician that has to improvise, and I know that this has been a key skill that I have perfected. Of course being aware that this is how I operate, I will often forcibly change it to bring something new to the table.
Each story speaks in a different way and you have to listen for the instructions it feeds you.