I just upgraded to 6.3.4 and confirmed that after upgrade it "remembers" the location of:
PCGen Character Directory
PCGen Custom Data Directory
PCGen Homebrew Data Directory
PCGen Portraits Directory
Not sure if this is the behavior that was intended.
Note that my options.ini is not stored in the PCGen directory.
This one has already been reported.