I'm starting a new tank with a Betta and a African Dwarf Frog, any suggestions?

The frog and the Betta should do fine together. I have the same setup in a 12 gal. with a cory too. Stress coat should not hurt them but it is not really needed. My advice is to treat the tank with some Amquel preferably not Amquel plus but the original since you can't really overdo it with the original. Right now I feed all the tank mates a good flake food alternated with freeze dried mysis shrimp, and they all seem to like it and are doing well. You have to be careful with live plants since they have additional light and CO2 requirements. In a tank that small I would stick to fake plants. The pH should be around 7. I would heat the tank, but it your room temp is stable and consistently in the 72-76 range you should be fine. If it is not invest in a small submersible heater and thermometer. As for the current most filters have some sort of level or dial that restricts the flow, if yours does not then I would invest in a different filter.